Pith. sign in

REVIEW 3 major objections 5 minor 65 references

MultimodalHugs: Enabling Sign Language Processing in Hugging Face

T0 review · 3 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A six-field TSV format plus configuration file lets sign-language translation run on Hugging Face with no custom training code.

desk verdict A useful framework idea undermined by overclaimed evidence: the only executed reproduction shows an unexplained BLEU gap, and the SOTA 'reproduction' was never actually run. read the letter →

arxiv 2509.09729 v1 pith:3PXS5Y5W submitted 2025-09-10 cs.CL cs.AIcs.MM

Gerard Sant , Zifan Jiang , Carlos Escolano , Amit Moryossef , Mathias Müller , Rico Sennrich , Sarah Ebling This is my paper · ORCID
classification cs.CLcs.AIcs.MM
keywords signlanguageprocessingmultimodaltranslationHuggingFacereproducibilityposeestimationpixel-basedmachineTSVdatasetformatframework
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that sign language processing lags behind text NLP because of tooling, not algorithms: experiments rest on ad-hoc scripts that are hard to reproduce. MultimodalHugs adds a standardization layer to the Hugging Face ecosystem, describing every example as one TSV row and every training run as one YAML file, so switching inputs—pose sequences, precomputed video features, raw video, or pixel-rendered words—means changing a processor and a config, not writing new code. The authors back this with two demonstrations: they reproduce a published pose-to-text sign language system and extend it to I3D features and end-to-end video, and they apply the same pipeline to Hebrew-to-English translation from images of words, matching a tokenizer-based baseline with 45% fewer tokens. If the schema is as expressive as claimed, SLP gains a common substrate for reproducible, comparable experiments.

What carries the argument

The load-bearing object is the six-field TSV row schema combined with the modular processor abstraction. The TSV forces every dataset into one shape so a single Hugging Face Trainer loop can run any task; the processor converts whatever the signal field points to (e.g., a .pose file, .mp4, or image sequence) into the model's input tensors. A YAML config instantiates the training actors—dataset, processor, model—and selects the model's components (feature extractor, multimodal mapper, backbone), so setup, training, and generation are three command-line calls.

What would settle it

Run the Appendix C reproduction of Zhang et al. (2024) to completion—SLT, alignment, machine translation, and augmented SLT—from only the TSV and YAML files and compare BLEU/chrF with the published numbers. The paper states this run was never completed, so any divergence, required custom code, or schema limit (e.g., a sample needing more than one embedded signal) would disprove the central claim.

Watch

Extended reading notes

Core claim

The paper's central claim is that a fixed TSV schema—signal, signal_start, signal_end, encoder_prompt, decoder_prompt, output—plus a YAML configuration is enough to express a wide class of sign language and multimodal translation experiments, so a single untampered training loop can handle pose data, video features, raw frames, and rendered text. The processor abstraction decouples modality-specific preprocessing from the model, making a modality or backbone swap a configuration change rather than a code-writing task. Evidence includes a faithful reproduction of the Uthus et al. (2023) pose-to-text system, improved results when pose is replaced by I3D features, and a pixel-based Hebrew-to-En

Load-bearing premise

The six-field TSV schema plus the available processors is assumed expressive enough to capture any realistic sign language experiment; if a task needs multiple simultaneous signals or an alignment the schema cannot represent, the 'no additional code' promise fails.

Editorial extensions

If this is right

  • A published SLP system can be re-run by preparing TSV files and a YAML file, with no custom training scripts.
  • Modality comparisons (pose vs. I3D features vs. raw video) become cheap and fair, since only the processor and dataset pointers change.
  • The same infrastructure extends beyond sign languages to tasks where input can be rendered as frames, such as pixel-based translation for morphologically rich scripts.
  • By inheriting Hugging Face's Trainer, the framework brings mixed-precision, distributed training, and standard metrics to SLP experiments.
  • Full automation of setup/train/generate makes the entire experiment—data, preprocessing, and evaluation—auditable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The schema allows only one primary signal per example, so a task needing simultaneous or interleaved modalities (e.g., pose plus video in one input) still requires custom code today; the paper's own future-work section points to this limit.
  • The Hebrew pixel experiment suggests visual tokenization could be a general workaround for morphology-heavy languages, providing an efficiency lever (45% fewer tokens) beyond just sign languages.
  • If the TSV format becomes conventional, dataset-level comparison improves and the 'code not published' failure mode the paper surveys becomes easy to audit, since a TSV file itself is the interface.
  • The universality of 'no additional code' is bounded by the available processor library: each genuinely new modality requires a new processor, so the framework's reach grows with that library.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces MultimodalHugs, a framework built on Hugging Face that aims to make multimodal sign language processing (SLP) reproducible and easy by standardizing datasets as TSV files, using modular modality-specific processors, and controlling training/evaluation through YAML configuration and three CLI commands (setup, train, generate). The authors argue that current SLP code is ad hoc and that Hugging Face lacks support for non-text inputs. They demonstrate the framework on two tasks: (1) sign-to-text translation on How2Sign using MediaPipe poses, I3D features, and raw video, with mT5 and M2M-100 backbones; and (2) Hebrew-to-English translation where Hebrew words are rendered as pixel images and embedded with CLIP. Appendix C claims to reproduce the multitask SOTA system of Zhang et al. (2024) without custom code, but explicitly states the experiment was not run to completion. The central claim of a general no-custom-code framework is therefore backed by two relatively simple executed demonstrations and an unexecuted complex reproduction.

Significance. If the framework performs as claimed, it would address a real and widely felt need in SLP: reducing ad-hoc code, enabling fairer comparisons, and integrating with the Hugging Face ecosystem. The paper's strengths include a clean conceptual abstraction (TSV + processors + YAML), open-source release, multiple executed experiments spanning different modalities, and transparent reporting in Appendix A. However, two issues affect the central claim. First, the only executed comparison with prior work (§4.1, Table 1) shows an unexplained BLEU gap: the supposed faithful reproduction of Uthus et al. gives 1.43 BLEU versus the original 0.86. Second, the strongest evidence for the 'no additional code' claim—the SOTA reproduction in Appendix C—was not actually run, so the four-task multitask setup and its mixing/weighting/evaluation remain unverified. These gaps are fixable but currently undermine the paper's most prominent promise.

major comments (3)
  1. [§4.1, Table 1] The reproduction claim for Uthus et al. (2023) is not supported by the reported numbers. Row 0 reports the original result of BLEU 0.86; row 1 reports the MultimodalHugs reproduction at BLEU 1.43, a roughly 66% relative improvement. The text states this 'replicates the original system' without any explanation for the gap. If the same data splits, preprocessing, evaluation script, and checkpoint selection were used, the discrepancy itself needs explanation. If the setup differs in any way (e.g., use of How2Sign versus other data, tokenization, case handling, beam size, seed, or epoch count), those differences must be stated. As written, this undermines the 'faithful reproduction' claim and affects the credibility of the other rows.
  2. [Appendix C.3, §3.1] The claim that MultimodalHugs can 'simply reproduce' the state-of-the-art multitask SLT system of Zhang et al. (2024) without custom code is not experimentally verified. Appendix C.3 explicitly states: 'Although we did not run this experiment to completion due to computational constraints...'. The appendix shows how the four tasks (SLT, alignment, MT, augmented SLT) can be mapped to the TSV fields, but it does not demonstrate that the seq2seq task actually handles the four distinct objectives, their mixing, weighting, or evaluation. Since this is the only complex multitask demonstration in the paper, the 'without requiring any custom scripts' claim is unsupported. I recommend either running at least a reduced-scale or component-wise version of the multitask experiment, or explicitly re-scoping the claim to 'designed to support' rather than 'reproduces'.
  3. [§3.1, §4] The paper's core expressiveness claim—that a fixed six-field TSV schema plus the modular processor abstraction can capture 'essentially any' multimodal SLP experiment without task-specific code—is demonstrated only on two tasks: pose/video-to-text SLT and Hebrew pixel-based MT. Both tasks share the same input modality (a single signal sequence) and text output. The framework's ability to handle multiple simultaneous signals (e.g., pose + video), non-text outputs, or tasks requiring alignment losses is not shown. This is closely tied to the unexecuted Appendix C: the most complex scenario that would exercise these capabilities is precisely the one that was not run. Please either provide at least one additional executed experiment with a structurally different input/output configuration, or temper the generality claims in §1 and §3.1.
minor comments (5)
  1. [Abstract, §3.1] Grammar: 'Appendix C shows how MultimodalHugs simply reproduction of a complex...' should be 'simply reproduces'. Similar phrasing in the abstract could be tightened.
  2. [§3.4] There is a garbled line of characters immediately after the multimodalhugs-generate code block ('.אר×פשוטØÚ'). This appears to be a rendering/encoding artifact and should be removed.
  3. [References] Author name formatting issues: 'Leandro V on Werra' should be 'Leandro Von Werra'; the entry for 'Yin Kayo and Read Jesse' should be alphabetized under 'Yin' and formatted as 'Kayo Yin and Jesse Read'.
  4. [Appendix A] The survey is small (n=16) and self-selected; the use of phrases such as 'strong interest' and 'would consider using it as their main framework' should be framed as informal usability feedback rather than a generalizable empirical result. The summary already calls it 'informal', which is good, but the main text should mirror this caveat.
  5. [Table 2] The metric 'XCOMET-XXL' is used without a citation or definition in the text. If it is a version of COMET, please cite the corresponding reference and specify which checkpoint was used.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the framework's claims are empirical/engineering assertions, and the unexecuted Appendix C reproduction and BLEU gap are verification gaps, not reductions to inputs.

full rationale

The paper is a software-framework contribution, not a derivation whose conclusions are fed back as assumptions. The two demonstrations are compared against external baselines (Uthus et al. 2023 for SLT; M2M-100 for Hebrew MT) using standard metrics, and the results do not reduce to the framework's own design choices. The SLT 'reproduction' is a training run on the same How2Sign data with the same linear-mT5 architecture; although Table 1 row 1 (BLEU 1.43) does not match row 0 (0.86), that is an unexplained discrepancy or reproducibility gap, not a circular reduction—the evaluation is independent of the framework's own predictions. Appendix C, which claims that the Zhang et al. SOTA system can be replicated with TSV+YAML, explicitly states: 'Although we did not run this experiment to completion due to computational constraints, this setup faithfully replicates the architecture and multitask training scheme described by Zhang et al. (2024), relying solely on standardized TSV metadata and a YAML configuration file.' That is an unexecuted verification claim, not a definitional identity. The survey in Appendix A is self-collected and supports the motivational premise, but it is not used as a formal premise in any claimed result, so it does not make the derivation circular. Self-citations (e.g., Müller et al. 2023a,b; Moryossef et al. 2023) point to external tools or prior findings and are corroborated by the paper's own experiments. No load-bearing claim reduces by construction to its own input.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

No free parameters are fitted to data; all results use standard pretrained models and fixed hyperparameters. The framework introduces no new theoretical entities. The main assumption is the sufficiency of the TSV abstraction, which is a domain assumption about data representation, not a mathematical axiom.

assumptions (1)
  • domain assumption The TSV schema (signal, signal_start/end, encoder_prompt, decoder_prompt, output) and generic processors are expressive enough to represent any multimodal SLP experiment without task-specific code.
    This is the core design assumption introduced in §3.1 and used to claim general applicability, but it is only demonstrated on two tasks. There may be SLP data or tasks that do not fit this schema (e.g., multiple simultaneous inputs, non-text outputs), as the survey respondents themselves noted (§A.1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of MultimodalHugs: Enabling Sign Language Processing in Hugging Face." pith.science (2026). https://pith.science/paper/3PXS5Y5W

@misc{pith2026250909729,
  author       = {Pith},
  title        = {Pith review of: MultimodalHugs: Enabling Sign Language Processing in Hugging Face},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3PXS5Y5W}},
  note         = {Machine review of arXiv:2509.09729}
}
read the original abstract

In recent years, sign language processing (SLP) has gained importance in the general field of Natural Language Processing. However, compared to research on spoken languages, SLP research is hindered by complex ad-hoc code, inadvertently leading to low reproducibility and unfair comparisons. Existing tools that are built for fast and reproducible experimentation, such as Hugging Face, are not flexible enough to seamlessly integrate sign language experiments. This view is confirmed by a survey we conducted among SLP researchers. To address these challenges, we introduce MultimodalHugs, a framework built on top of Hugging Face that enables more diverse data modalities and tasks, while inheriting the well-known advantages of the Hugging Face ecosystem. Even though sign languages are our primary focus, MultimodalHugs adds a layer of abstraction that makes it more widely applicable to other use cases that do not fit one of the standard templates of Hugging Face. We provide quantitative experiments to illustrate how MultimodalHugs can accommodate diverse modalities such as pose estimation data for sign languages, or pixel data for text characters.

Figures

Figures reproduced from arXiv: 2509.09729 by the authors.

Figure 1
Figure 1. Illustration of the primary modalities used [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the MultimodalHugs workflow. The figure illustrates the four core steps: a) dataset standardization using TSV files (ASL: American Sign Language); b) setup of training configurations and modules; c) streamlined model training; and d) evaluation through automatic generation and metrics. specific modifications. For instance, numeric pose sequences or video frames cannot directly leverage tokenizer-based pr… view at source ↗
Figure 3
Figure 3. Schematic overview of the MultimodalHugs processing pipeline, covering all experiments presented in both demonstrations. Sequences can come from different modalities, each processed by a dedicated processor to produce the appropriate input tensors for the model. The right side illustrates the translation training workflow, where text labels are also processed and provided as targets. multimodalhugs-generate --task <… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Example training pipeline for a pose-based experiment in [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: Multitask strategy proposed by Zhang et al. (2024). The figure illustrates an encoder-decoder SLT model trained on four pretraining tasks, all implemented as sequence-to-sequence learning within a shared architecture. Input prompts (red), sign frames (green), and targe…
Figure 6
Figure 6. Figure 6: Envisioned future training pipeline for mixed-modality sequences in [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

65 extracted references · 10 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Sadam Al-Azani and El-Sayed M. El-Alfy. 2017. https://doi.org/10.1016/j.procs.2017.05.365 Using word embedding and ensemble learning for highly imbalanced data sentiment analysis in short arabic text . Procedia Computer Science, 109:359--366. 8th International Conference on Ambient Systems, Networks and Technologies, ANT-2017 and the 7th International Con...

    work page doi:10.1016/j.procs.2017.05.365 2017

  4. [4]

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob L Menick, Sebastian Borgeaud, Andy Brock, Aida Nematzadeh, Sahand Sharifzadeh, Miko aj Bi\' n...

    2022

  5. [5]

    Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, Jenia Jitsev, Simon Kornblith, Pang Wei Koh, Gabriel Ilharco, Mitchell Wortsman, and Ludwig Schmidt. 2023. Openflamingo: An open-source framework for training large autoregressive vision-language models. arXiv prepri...

    arXiv 2023

  6. [6]

    Murat Aydoğan and Ali Karci. 2020. https://doi.org/10.1016/j.physa.2019.123288 Improving the accuracy using pre-trained word embeddings on deep neural networks for turkish text classification . Physica A: Statistical Mechanics and its Applications, 541:123288

    arXiv 2020

  7. [7]

    Alessia Battisti, Emma van den Bold, Anne G \"o hring, Franz Holzknecht, and Sarah Ebling. 2024. https://aclanthology.org/2024.signlang-1.2/ Person identification from pose estimates in sign language . In Proceedings of the LREC-COLING 2024 11th Workshop on the Representation and Processing of Sign Languages: Evaluation of Sign Language Resources, pages 1...

    2024

  8. [8]

    Anya Belz, Shubham Agarwal, Anastasia Shimorina, and Ehud Reiter. 2021. https://doi.org/10.18653/v1/2021.eacl-main.29 A systematic review of reproducibility research in natural language processing . In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 381--393, Online. Associati...

    doi:10.18653/v1/2021.eacl-main.29 2021

Show all 65 references
  1. [9]

    Anya Belz and Craig Thomson. 2023. https://aclanthology.org/2023.humeval-1.4/ The 2023 R epro NLP shared task on reproducibility of evaluations in NLP : Overview and results . In Proceedings of the 3rd Workshop on Human Evaluation of NLP Systems, pages 35--48, Varna, Bulgaria....

    2023

  2. [10]

    Danielle Bragg, Oscar Koller, Mary Bellard, Larwan Berke, Patrick Boudreault, Annelies Braffort, Naomi Caselli, Matt Huenerfauth, Hernisa Kacorri, Tessa Verhoef, Christian Vogler, and Meredith Ringel Morris. 2019. https://doi.org/10.1145/3308561.3353774 Sign Language Recogniti...

    2019

  3. [11]

    Necati Cihan Camgoz, Oscar Koller, Simon Hadfield, and Richard Bowden. 2020. Sign language transformers: Joint end-to-end sign language recognition and translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2020

  4. [12]

    Necati Cihan Camgöz, Ben Saunders, Guillaume Rochette, Marco Giovanelli, Giacomo Inches, Robin Nachtrab-Ribback, and Richard Bowden. 2021. https://doi.org/10.1109/FG52635.2021.9667087 Content4all open research sign language translation datasets . In 2021 16th IEEE Internationa...

    2021

  5. [13]

    Zhe Cao, Gines Hidalgo, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2021. https://doi.org/10.1109/TPAMI.2019.2929257 Openpose: Realtime multi-person 2d pose estimation using part affinity fields . IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(1):172--186

    2021

  6. [14]

    Joao Carreira and Andrew Zisserman. 2017. Quo vadis, action recognition? a new model and the kinetics dataset. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    2017

  7. [15]

    Yutong Chen, Ronglai Zuo, Fangyun Wei, Yu Wu, Shujie LIU, and Brian Mak. 2022. https://proceedings.neurips.cc/paper_files/paper/2022/file/6cd3ac24cdb789beeaa9f7145670fcae-Paper-Conference.pdf Two-stream network for sign language recognition and translation . In Advances in Neu...

    2022

  8. [16]

    Avihay Chriqui and Inbal Yahav. 2022. https://doi.org/10.1287/ijds.2022.0016 Hebert and hebemo: A hebrew bert model and a tool for polarity analysis and emotion recognition . INFORMS Journal on Data Science, 1(1):81--95

    2022

  9. [17]

    Mathieu De Coster, Karel D ' Oosterlinck, Marija Pizurica, Paloma Rabaey, Severine Verlinden, Mieke Van Herreweghe, and Joni Dambre. 2021. https://aclanthology.org/2021.mtsummit-at4ssl.10/ Frozen pretrained transformers for neural sign language translation . In Proceedings of ...

    2021

  10. [18]

    Jesse Dodge. 2020. Towards Efficient and Reproducible Natural Language Processing. Ph.D. thesis, Carnegie Mellon University

    2020

  11. [19]

    Amanda Duarte, Shruti Palaskar, Lucas Ventura, Deepti Ghadiyaram, Kenneth DeHaan, Florian Metze, Jordi Torres, and Xavier Giro-i Nieto. 2021. How2sign: A large-scale multimodal dataset for continuous american sign language. In Proceedings of the IEEE/CVF Conference on Computer...

    2021

  12. [20]

    Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, Naman Goyal, Tom Birch, Vitaliy Liptchinsky, Sergey Edunov, Michael Auli, and Armand Joulin. 2021. http://jmlr.org/papers/v...

    2021

  13. [21]

    Hossam Faris et al. 2020. https://www.scitepress.org/Papers/2020/89540/89540.pdf Hate speech detection using word embedding and deep learning in the arabic language context . In ICPRAM, pages 453--460

    2020

  14. [22]

    Awet Fesseha, Shengwu Xiong, Eshete Derb Emiru, Moussa Diallo, and Abdelghani Dahou. 2021. https://doi.org/10.3390/info12020052 Text classification based on convolutional neural networks and word embedding for low-resource languages: Tigrinya . Information, 12(2)

    2021 doi

  15. [23]

    Mohammed M Fouad, Ahmed Mahany, Naif Aljohani, Rabeeh Ayaz Abbasi, and Saeed-Ul Hassan. 2020. https://link.springer.com/article/10.1007/s00500-019-04153-6 Arwordvec: efficient word embedding models for arabic tweets . Soft Computing, 24(11):8061--8068

    2020 doi

  16. [24]

    Ivan Grishchenko and Valentin Bazarevsky. 2020. https://ai.googleblog.com/2020/12/mediapipe-holistic-simultaneous-face.html MediaPipe Holistic - simultaneous face, hand and pose prediction, on device . Google AI Blog

    2020

  17. [25]

    Jelinek, R

    F. Jelinek, R. L. Mercer, L. R. Bahl, and J. K. Baker. 2005. https://doi.org/10.1121/1.2016299 Perplexity—a measure of the difficulty of speech recognition tasks . The Journal of the Acoustical Society of America, 62(S1):S63--S63

    2005 doi

  18. [26]

    Abhinav Joshi, Romit Mohanty, Mounika Kanakanti, Andesha Mangla, Sudeep Choudhary, Monali Barbate, and Ashutosh Modi. 2024. https://doi.org/10.18653/v1/2024.findings-acl.643 i S ign: A benchmark for I ndian S ign L anguage processing . In Findings of the Association for Comput...

    2024 doi

  19. [27]

    Tarek Kanan, Odai Sadaqa, Amal Aldajeh, Hanadi Alshwabka, Wassan AL-dolime, Shadi AlZu’bi, Mohammed Elbes, Bilal Hawashin, and Mohammad A. Alia. 2019. https://doi.org/10.1109/JEEIT.2019.8717369 A review of natural language processing and machine learning tools used to analyze ...

    2019

  20. [28]

    Yin Kayo and Read Jesse. 2020. https://doi.org/10.18653/v1/2020.coling-main.525 Better sign language translation with stmc-transformer . Proceedings of the 28th International Conference on Computational Linguistics

    2020 doi

  21. [29]

    Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander Rush. 2017. https://aclanthology.org/P17-4012/ O pen NMT : Open-source toolkit for neural machine translation . In Proceedings of ACL 2017, System Demonstrations , pages 67--72, Vancouver, Canada. Associati...

    2017

  22. [30]

    Stav Klein and Reut Tsarfaty. 2020. https://doi.org/10.18653/v1/2020.sigmorphon-1.24 Getting the \# \# life out of living: How adequate are word-pieces for modelling complex morphology? In Proceedings of the 17th SIGMORPHON Workshop on Computational Research in Phonetics, Phon...

    2020 doi

  23. [31]

    Sneha Kudugunta, Isaac Caswell, Biao Zhang, Xavier Garcia, Derrick Xin, Aditya Kusupati, Romi Stella, Ankur Bapna, and Orhan Firat. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/d49042a5d49818711c401d34172f9900-Paper-Datasets_and_Benchmarks.pdf Madlad-400: A...

    2023

  24. [32]

    Elmurod Kuriyozov, Yerai Doval, and Carlos G \'o mez-Rodr \'i guez. 2020. https://aclanthology.org/2020.lrec-1.499/ Cross-lingual word embeddings for T urkic languages . In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4054--4062, Marseille, Fr...

    2020

  25. [33]

    Md Tahmid Rahman Laskar, Sawsan Alqahtani, M Saiful Bari, Mizanur Rahman, Mohammad Abdullah Matin Khan, Haidar Khan, Israt Jahan, Amran Bhuiyan, Chee Wei Tan, Md Rizwan Parvez, Enamul Hoque, Shafiq Joty, and Jimmy Huang. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.764 A ...

    2024 doi

  26. [34]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/6dcf277ea32ce3288914faf369fe6de0-Paper-Conference.pdf Visual instruction tuning . In Advances in Neural Information Processing Systems, volume 36, pages 34...

    2023

  27. [35]

    Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, Chris McClanahan, Esha Uboweja, Michael Hays, Fan Zhang, Chuo-Ling Chang, Ming Guang Yong, Juhyun Lee, Wan-Teh Chang, Wei Hua, Manfred Georg, and Matthias Grundmann. 2019. https://arxiv.org/abs/1906.08172 Mediapipe: A framework for ...

    2019 arXiv

  28. [36]

    Amit Moryossef, Mathias M \" u ller, and Rebecka Fahrni. 2023. https://doi.org/10.48550/ARXIV.2310.09066 pose-format: Library for viewing, augmenting, and handling .pose files . CoRR, abs/2310.09066

    2023 doi arXiv

  29. [37]

    u ller, Sarah Ebling, Eleftherios Avramidis, Alessia Battisti, Mich \`e le Berger, Richard Bowden, Annelies Braffort, Necati Cihan Camg \

    Mathias M \"u ller, Sarah Ebling, Eleftherios Avramidis, Alessia Battisti, Mich \`e le Berger, Richard Bowden, Annelies Braffort, Necati Cihan Camg \"o z, Cristina Espa \ n a-bonet, Roman Grundkiewicz, Zifan Jiang, Oscar Koller, Amit Moryossef, Regula Perrollaz, Sabine Reinhar...

    2022

  30. [38]

    Mathias M \"u ller, Zifan Jiang, Amit Moryossef, Annette Rios, and Sarah Ebling. 2023 a . https://doi.org/10.18653/v1/2023.acl-short.60 Considerations for meaningful sign language machine translation based on glosses . In Proceedings of the 61st Annual Meeting of the Associati...

    2023 doi

  31. [39]

    Mathias M \"u ller, Annette Rios, Amit Moryossef, Sarah Ebling, and Thomas Hanke UHH. 2023 b . https://www.project-easier.eu/wp-content/uploads/sites/67/2023/08/EASIER_D4.3_Translation-models-final-version-v.1.2.pdf D4.3 easier final translation systems v2

    2023

  32. [40]

    Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. https://doi.org/10.18653/v1/N19-4009 fairseq: A fast, extensible toolkit for sequence modeling . In Proceedings of the 2019 Conference of the North A merican Chap...

    2019 doi

  33. [41]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. https://doi.org/10.3115/1073083.1073135 B leu: a method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311--3...

    2002

  34. [42]

    Maja Popovi \'c . 2015. https://doi.org/10.18653/v1/W15-3049 chr F : character n-gram F -score for automatic MT evaluation . In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392--395, Lisbon, Portugal. Association for Computational Linguistics

    2015 doi

  35. [43]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. https://proceedings.mlr.press/v139/radford21a.html Learning transferable visual model...

    2021

  36. [44]

    Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.213 COMET : A neural framework for MT evaluation . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685--2702, ...

    2020 doi

  37. [45]

    Lotz, Emanuele Bugliarello, Elizabeth Salesky, Miryam de Lhoneux, and Desmond Elliott

    Phillip Rust, Jonas F. Lotz, Emanuele Bugliarello, Elizabeth Salesky, Miryam de Lhoneux, and Desmond Elliott. 2023. https://openreview.net/forum?id=FkSp8VW8RjH Language modelling with pixels . In The Eleventh International Conference on Learning Representations, ICLR 2023, Kig...

    2023

  38. [46]

    Holger Schwenk, Guillaume Wenzek, Sergey Edunov, Edouard Grave, and Armand Joulin. 2020. https://arxiv.org/abs/1911.04944 Ccmatrix: Mining billions of high-quality parallel sentences on the web . Preprint, arXiv:1911.04944

    2020 arXiv

  39. [47]

    Amit Seker, Elron Bandel, Dan Bareket, Idan Brusilovsky, Refael Greenfeld, and Reut Tsarfaty. 2022. https://doi.org/10.18653/v1/2022.acl-long.4 A leph BERT : Language model pre-training and evaluation from sub-word to sentence level . In Proceedings of the 60th Annual Meeting ...

    2022 doi

  40. [48]

    Telem Joyson Singh, Sanasam Ranbir Singh, and Priyankoo Sarmah. 2023. https://aclanthology.org/2023.paclic-1.69 Subwords to word back composition for morphologically rich languages in neural machine translation . In Proceedings of the 37th Pacific Asia Conference on Language, ...

    2023

  41. [49]

    Valerie Sutton. 1990. Lessons in sign writing. SignWriting

    1990

  42. [50]

    S. Tan, N. Khan, Z. An, Y. Ando, and R. Kawakami. 2024. https://doi.org/10.1080/01691864.2024.2442721 A review of deep learning-based approaches to sign language processing . Advanced Robotics, 38(23):1649--1667(19)

    2024

  43. [51]

    Garrett Tanzer and Biao Zhang. 2024. https://arxiv.org/abs/2407.11144 Youtube-sl-25: A large-scale, open-domain multilingual sign language parallel corpus . Preprint, arXiv:2407.11144

    2024 arXiv

  44. [52]

    G\'allego, Amanda Duarte, Jordi Torres, and Xavier Gir\'o-i Nieto

    Laia Tarr\'es, Gerard I. G\'allego, Amanda Duarte, Jordi Torres, and Xavier Gir\'o-i Nieto. 2023. https://openaccess.thecvf.com/content/CVPR2023W/WiCV/html/Tarres_Sign_Language_Translation_from_Instructional_Videos_CVPRW_2023_paper.html Sign language translation from instructi...

    2023

  45. [53]

    NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prang...

    2022 arXiv

  46. [54]

    Yingli Tian, Jianbo Su, Lan Ni, and Yuchun Fang. 2024. https://doi.org/10.1142/S2972335324010038 Bridging the gap: Ai and sign language recognition– a path toward inclusive communication . International Journal of Artificial Intelligence and Robotics Research, 0(0):2401003

    2024 doi

  47. [55]

    Reut Tsarfaty, Djam \'e Seddah, Yoav Goldberg, Sandra Kuebler, Yannick Versley, Marie Candito, Jennifer Foster, Ines Rehbein, and Lamia Tounsi. 2010. https://aclanthology.org/W10-1401/ Statistical parsing of morphologically rich languages ( SPMRL ) what, how and whither . In P...

    2010

  48. [56]

    Dave Uthus, Garrett Tanzer, and Manfred Georg. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/5c61452daca5f0c260e683b317d13a3f-Paper-Datasets_and_Benchmarks.pdf Youtube-asl: A large-scale, open-domain american sign language-english parallel corpus . In Advanc...

    2023

  49. [57]

    Leandro Von Werra, Lewis Tunstall, Abhishek Thakur, Sasha Luccioni, Tristan Thrush, Aleksandra Piktus, Felix Marty, Nazneen Rajani, Victor Mustar, and Helen Ngo. 2022. https://doi.org/10.18653/v1/2022.emnlp-demos.13 Evaluate & evaluation on the hub: Better best practices for d...

    2022 doi

  50. [58]

    Waghmare and Ashwini M

    Prachi P. Waghmare and Ashwini M. Deshpande. 2023. https://ijritcc.org/index.php/ijritcc/article/view/9765/7458 A study on techniques and challenges in sign language translation . International Journal on Recent and Innovation Trends in Computing and Communication, 11(9):4039-...

    2023

  51. [59]

    Philip Whittington, Gregor Bachmann, and Tiago Pimentel. 2024. https://arxiv.org/abs/2412.15210 Tokenisation is np-complete . Preprint, arXiv:2412.15210

    2024 arXiv

  52. [60]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020 doi

  53. [61]

    Linting Xue, Aditya Barua, Noah Constant, Rami Al - Rfou, Sharan Narang, Mihir Kale, Adam Roberts, and Colin Raffel. 2022. https://doi.org/10.1162/TACL\_A\_00461 Byt5: Towards a token-free future with pre-trained byte-to-byte models . Trans. Assoc. Comput. Linguistics, 10:291--306

    2022 doi

  54. [62]

    Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. https://arxiv.org/abs/2010.11934 mt5: A massively multilingual pre-trained text-to-text transformer . Preprint, arXiv:2010.11934

    2021 arXiv

  55. [63]

    Kayo Yin, Amit Moryossef, Julie Hochgesang, Yoav Goldberg, and Malihe Alikhani. 2021. https://doi.org/10.18653/v1/2021.acl-long.570 Including signed languages in natural language processing . In Proceedings of the 59th Annual Meeting of the Association for Computational Lingui...

    2021 doi

  56. [64]

    Biao Zhang, Garrett Tanzer, and Orhan Firat. 2024. https://arxiv.org/abs/2407.11855 Scaling sign language translation . Preprint, arXiv:2407.11855

    2024 arXiv

  57. [65]

    Ce Zheng, Wenhan Wu, Chen Chen, Taojiannan Yang, Sijie Zhu, Ju Shen, Nasser Kehtarnavaz, and Mubarak Shah. 2023. https://doi.org/10.1145/3603618 Deep learning-based human pose estimation: A survey . ACM Comput. Surv., 56(1)

    2023 doi

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.