REVIEW 3 major objections 5 minor 45 references
A HamNoSys-Guided Dataset and Baselines for Fine-Grained Isolated Handshape Recognition in Sign Language
T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read The paper claims a balanced HamNoSys-grounded handshape benchmark is reproducible and that generalising to unseen signers, not within-participant recognition, is the main challenge.
desk verdict Genuinely useful benchmark resource; the chart-to-label mapping is the soft spot, and it isn't publicly released. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the official, non-exhaustive HamNoSys 4 Handshapes Chart: a publicly illustrated reference inventory of hand models used to define a bounded set of 160 classes, with each drawn hand treated as one class and blank or symbol-only cells excluded. Around this, the acquisition pipeline records 10-second clips with a prescribed hand rotation and extracts every fifth frame to yield 60 images per class per participant, giving 144,000 images. The evaluation machinery is the pairing of a class-stratified frame-level split with a 15-fold leave-one-subject-out protocol, which separates seen-participant reference performance from true signer-independent generalisation. A hand-landmark extractor from the literature supplies both the hand crops for the appearance models and the 21-point landmark graphs used by the landmark-based models.
What would settle it
Have HamNoSys specialists independently label a stratified sample of around 1,000 images from the modelling subset using the original chart illustrations without seeing the dataset's class codes, and measure agreement with those codes; if per-class agreement is well below the internal verification standard, label noise is large enough to change the reported accuracy gaps.
Extended reading notes
Core claim
The paper's central claim is that a balanced, real-image benchmark for fine-grained isolated handshape recognition can be built directly from the official HamNoSys 4 Handshapes Chart, and that this benchmark is needed because existing resources either cover only language-specific fingerspelling alphabets, use synthetic images, or carry weak corpus-derived labels. Treating each distinct illustrated hand model as one class yields 160 classes; a controlled acquisition protocol with operator verification produced 144,000 RGB images from 15 participants. Under a participant-overlapping split, the best model, ViT-B/16, reaches 86.20 percent top-1 accuracy, while under leave-one-subject-out evaluation the same model falls to 45.22 percent, and ResNet-18 falls to 45.38 percent. The conclusion the authors draw is that seen-participant recognition is no longer the limiting factor; unseen-participant generalisation is the principal challenge.
Load-bearing premise
The load-bearing premise is that every drawn hand model in the HamNoSys 4 Handshapes Chart is an unambiguous handshape class and that the operator's reference-guided verification ensured participants reproduced those classes, so the reported accuracies stand or fall with that label validity.
Editorial extensions
If this is right
- If the central claim is right, the HamNoSys chart can be used as a class inventory without expert transcription, making phonetically defined handshape resources reproducible across labs.
- If the central claim is right, participant-overlapping accuracy numbers overstate deployable performance, and leave-one-subject-out evaluation should become the default reporting protocol for signer-independent handshape recognition.
- If the central claim is right, the roughly 39-to-41 point drop between the two protocols quantifies how much of the task is signer identity rather than handshape identity.
- If the central claim is right, the documented confusion pairs give a concrete error taxonomy for future models to target finger selection, bending, and thumb opposition.
- If the central claim is right, matched-model results on LSWH100 and ASL Fingerspelling Dataset A provide external anchors, not claims about which dataset is intrinsically harder.
Reading between the lines
- An extension the authors leave implicit is that the 160-class inventory could serve as a pretraining task for downstream continuous sign-language recognition, and the per-participant metadata would allow testing whether such pretraining reduces the leave-one-subject-out gap.
- The near-balanced class distribution and participant identifiers make this resource usable for fairness and calibration audits across signers, though the paper itself does not perform those analyses.
- A testable prediction consistent with the confusion analysis is that synthetic or multi-camera augmentation targeted at the documented close-pair errors would improve unseen-signer accuracy more than generic augmentation.
- Because the chart is non-exhaustive, dynamic two-handed transitions are absent; extending the same class-mapping scheme to those forms would be a natural incremental benchmark rather than a redesign.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces a handshape-recognition dataset of 144,000 RGB images collected from 15 participants, with 160 classes derived from the illustrated HamNoSys 4 Handshapes Chart. Four baseline families (ResNet-18, ViT-B/16, a landmark GCN, and XGBoost) are evaluated under a class-stratified subject-dependent frame-level split and a 15-fold leave-one-subject-out protocol, with additional matched-model experiments on LSWH100 and ASL Fingerspelling Dataset A. The central claims are that the resource is balanced and HamNoSys-grounded, that the evaluation protocols separate seen-signer from unseen-signer performance, and that the baselines provide reproducible reference numbers for the new benchmark.
Significance. If the resource is made publicly available and the label inventory is validated, this would fill a real gap: a broad, phonetically defined handshape benchmark with real images and explicit signer-independent evaluation. The paper's strengths include the systematic chart-to-class mapping procedure, the balanced per-class distribution, the clear separation between subject-dependent and LOSO protocols, and the use of four diverse baselines with matched external datasets. The moderately large gap between subject-dependent and LOSO results (e.g., 86.20% vs. 45.22% for ViT-B/16) is a useful and credible finding about the difficulty of unseen-signer handshape recognition. However, the central resource is not currently publicly inspectable, the label inventory rests on a single non-specialist operator with no reported agreement metric, and the subject-dependent split may be inflated by frame-level leakage from the same recording clips.
major comments (3)
- [§3.1–3.2 and Limitations] The 160-class inventory is defined by treating each drawn model in the HamNoSys chart as a class, and Section 3.2 states that participant reproductions were verified only by 'an operator with sign-language experience.' No second annotator, inter-annotator agreement statistic, or HamNoSys-specialist audit is reported; the Limitations section concedes that 'further annotation validation by HamNoSys specialists may strengthen the resource.' This is load-bearing because every reported accuracy is measured against these labels. A systematic misreading of subtle distinctions, such as the thumb-opposition forms that appear as the most confused pairs in Figure 9, would mean the benchmark is not measuring the intended classes. Please add a specialist audit or a documented inter-annotator reliability study, and release the class-mapping file so the chart-to-class correspondence can be independently checked.
- [§3.6.1] The subject-dependent protocol is a class-stratified random frame-level split applied after retaining every fifth frame from each 10-second clip. Because frames from the same source clip can appear simultaneously in training, validation, and test partitions, models can exploit clip-level appearance (background, lighting, identity, exact hand pose progression) rather than generalizable handshape information. This likely inflates the Table 9 accuracies and weakens the claim that these numbers serve as a 'seen-participant reference.' Please report an additional clip-disjoint or recording-disjoint subject-dependent split, or at least quantify the leakage effect by comparing against a split in which all frames from any given clip are kept in a single partition.
- [Data Availability] The dataset and the supplementary class-mapping file are only available 'upon reasonable request' from the corresponding author, and no public URL or repository is provided. For a dataset paper whose central contribution is the resource itself, this prevents independent verification of the chart-to-class mapping, label quality, and frame provenance, and it undermines the stated goal of providing a 'reproducible resource.' Please make the data, or at minimum the class-mapping file and a substantial annotated sample with metadata, publicly available under a clear license before acceptance; if full public release is impossible for institutional reasons, state the conditions explicitly and provide a persistent mechanism for access.
minor comments (5)
- [Abstract and body text] There are several typographical issues: the abstract contains 'pro-vide' with a line break, and the body has missing spaces such as 'overviewThe' and 'areproducibleresource'; please run a proofreading pass.
- [Limitations] The Limitations section begins with 'Few limitations should be noted'; this should read 'A few limitations should be noted.'
- [§4.1] The landmark preprocessing is described as 'preprocessed as in [43]' (the authors' prior ICPR paper). Please specify exactly which preprocessing steps are reused and whether any modifications were made, so that the baselines can be reproduced independently.
- [Table 6] The table reports 'Approx. per class' as 609, 130, and 131, but the actual per-class counts vary because 4,801 images were excluded from the modelling subset; consider reporting the min–max range per partition for clarity.
- [Figures 4 and 7] Figures 4 and 7 are dense and small at the current resolution; consider providing higher-resolution versions or vector graphics so the protocol details are legible.
Circularity Check
No significant circularity: the reported benchmarks are direct held-out measurements, and the only reused prior-work element is a non-load-bearing landmark preprocessing step.
full rationale
The paper's central deliverable is a dataset and baseline measurements, not a derivation from fitted quantities. The 160-class inventory is defined directly from the chart: 'Each distinct illustrated hand model in Fig. 1 was treated as one class; blank cells and cells containing only symbols or cross-references were excluded. This procedure produced 160 classes.' The reported accuracies are then measured on fixed labels under class-stratified subject-dependent splits and 15-fold leave-one-subject-out folds, with identical partitions across baselines; no fitted parameter is renamed as a prediction. The only reused prior-work component is the landmark preprocessing 'which were preprocessed as in [43]', where [43] is a prior paper by the same group; this is an input-formatting detail for two baselines and does not determine the measured accuracies or support the resource's central claim. No uniqueness theorem, imported ansatz, or self-definitional label/result equivalence appears. The paper's own limitation that 'further annotation validation by HamNoSys specialists may strengthen the resource' is a label-validity concern, not circularity: inaccurate chart-derived labels would degrade reported numbers, but the numbers would still be measurements. External matched-model results on LSWH100 and ASL Fingerspelling Dataset A further indicate that the evaluation pipeline is not defined circularly. The mild score reflects only the presence of one minor, non-load-bearing self-citation.
Assumptions & free parameters
assumptions (3)
- domain assumption Each distinct drawn hand model in the HamNoSys 4 Handshapes Chart defines a valid, mutually exclusive handshape class.
- domain assumption The operator's reference-guided verification ensures participants reproduced the intended handshape in every recording.
- domain assumption MediaPipe Hands detection is reliable enough that excluding the 4,801 undetected images does not bias the benchmark.
Cite this review
Pith. "Pith review of A HamNoSys-Guided Dataset and Baselines for Fine-Grained Isolated Handshape Recognition in Sign Language." pith.science (2026). https://pith.science/paper/IURM526R
@misc{pith2026260810588,
author = {Pith},
title = {Pith review of: A HamNoSys-Guided Dataset and Baselines for Fine-Grained Isolated Handshape Recognition in Sign Language},
year = {2026},
howpublished = {\url{https://pith.science/paper/IURM526R}},
note = {Machine review of arXiv:2608.10588}
}
read the original abstract
Purpose: Fine-grained handshape recognition supports computational sign-language transcription, recognition, and translation, but broad, phonetically defined visual inventories with signer-aware evaluation remain limited. This work introduces a benchmark grounded in the language-independent Hamburg Notation System (HamNoSys). Methods: A balanced dataset of 144,000 RGB images was collected from 15 participants for 160 handshape classes defined by the official HamNoSys 4 Handshapes Chart. ResNet-18 and ViT-B/16 were evaluated as appearance-based models, while a graph convolutional network and XGBoost were evaluated from hand landmarks. Both a class-stratified subject-dependent split and a 15-fold leave-one-subject-out (LOSO) protocol were used. The same model families were additionally assessed on LSWH100 and ASL Fingerspelling Dataset A for external context. Results: The subject-dependent benchmarks established reproducible reference performance across all four model families, whereas LOSO evaluation exposed a substantial reduction when recognition was required to generalise to unseen participants. On ASL Fingerspelling Dataset A, mean LOSO top-1 accuracy ranged from 82.20% to 87.40%. Conclusion: The documented acquisition, curation, and complementary evaluation protocols pro-vide a reproducible resource for fine-grained isolated-handshape research and for developing more accessible sign-language technologies.
Reference graph
Works this paper leans on
- [1]
-
[2]
Mitchell, R. E. & Young, T. A. How many people use sign language? a national health survey-based estimate.The Journal of Deaf Studies and Deaf Education28, 1–6 (2023)
work page 2023
-
[3]
Sandler, W. & Lillo-Martin, D.Sign Lan- guage and Linguistic Universals(Cambridge University Press, Cambridge, UK, 2006)
work page 2006
-
[4]
C.Sign Language Structure: An Outline of the Visual Communication Sys- tems of the American Deaf
Stokoe, W. C.Sign Language Structure: An Outline of the Visual Communication Sys- tems of the American Deaf. No. 8 in Studies in Linguistics: Occasional Papers (Depart- ment of Anthropology and Linguistics, Uni- versity of Buffalo, Buffalo, NY, 1960)
work page 1960
-
[5]
Rastgoo, R., Kiani, K. & Escalera, S. Sign language recognition: A deep sur- vey.Expert Systems with Applications164, 113794 (2021)
work page 2021
-
[6]
Bragg, D.et al.Sign language recognition, generation, and translation: An interdisci- plinary perspective. InProceedings of the 21st International ACM SIGACCESS Con- ference on Computers and Accessibility, 16– 31 (Association for Computing Machinery, New York, NY, USA, 2019)
work page 2019
-
[7]
De Coster, M., Shterionov, D., Van Her- reweghe, M. & Dambre, J. Machine transla- tion from signed to spoken languages: State of the art and challenges.Universal Access in the Information Society23, 1305–1331 (2024)
work page 2024
-
[8]
Garcia, B. & Sallandre, M.-A. Transcrip- tion systems for sign languages: A sketch of the different graphical representations of sign language and their characteristics. In Müller, C.et al.(eds.)Body–Language– Communication: An International Handbook on Multimodality in Human Interaction, 1125–1138 (De Gruyter Mouton, Berlin, Ger- many, 2013)
work page 2013
Show all 45 references
-
[9]
C., Xavier, A
Tkachman, O., Hall, K. C., Xavier, A. & Gick, B. Sign language phonetic annotation meets phonological CorpusTools: Towards a sign language toolset for phonetic notation and phonological analysis.Proceedings of the Annual Meetings on Phonology3(2016)
2016
-
[10]
Liddell, S. K. & Johnson, R. E. American sign language: The phonological base.Sign Language Studies195–277 (1989). 16
1989
-
[11]
& Brentari, D
Eccarius, P. & Brentari, D. Handshape coding made easier: A theoretically based notation for phonological transcription.Sign Language & Linguistics11, 69–101 (2008)
2008
-
[12]
Johnson, R. E. & Liddell, S. K. Toward a phonetic representation of signs, i: Sequen- tiality and contrast.Sign Language Studies 11, 241–274 (2011)
2011
-
[13]
C., Mackie, S., Fry, M
Hall, K. C., Mackie, S., Fry, M. & Tkachman, O. SLPAnnotator: Tools for implement- ing sign language phonetic annotation. In Proceedings of Interspeech 2017, 2083–2087 (2017)
2017
-
[14]
& Ebling, S
Jiang, Z., Moryossef, A., Müller, M. & Ebling, S. Machine translation between spoken languages and signed languages rep- resented in SignWriting. InFindings of the Association for Computational Linguis- tics: EACL 2023, 1706–1724 (Association for Computational Linguistics, Dub...
2023
-
[15]
Hochgesang, J. A. Using design principles to consider representation of the hand in some notation systems.Sign Language Studies14, 488–542 (2014)
2014
-
[16]
Dhanjal, A. S. & Singh, W. Comparative analysis of sign language notation systems for indian sign language. In2019 Second International Conference on Advanced Com- putational and Communication Paradigms (ICACCP), 1–6 (IEEE, Gangtok, India, 2019)
2019
-
[17]
& Henning, J.HamNoSys Version 2.0: Hamburg Notation System for Sign Lan- guages: An Introductory Guide, vol
Prillwitz, S., Leven, R., Zienert, H., Hanke, T. & Henning, J.HamNoSys Version 2.0: Hamburg Notation System for Sign Lan- guages: An Introductory Guide, vol. 5 of International Studies on Sign Language and Communication of the Deaf(Signum, Ham- burg, Germany, 1989)
1989
-
[18]
HamNoSys—representing sign language data in language resources and language processing contexts
Hanke, T. HamNoSys—representing sign language data in language resources and language processing contexts. In Stre- iter, O. & Vettori, C. (eds.)Proceedings of the LREC2004 Workshop on the Rep- resentation and Processing of Sign Lan- guages: From SignWriting to Image Pro- cess...
2004
-
[19]
HamNoSys 4 handshapes chart
Hanke, T. HamNoSys 4 handshapes chart. DGS-Korpus Project, University of Hamburg (2010). URL https://www.sign-lang.un i-hamburg.de/dgs-korpus/files/inhalt_pd f/HamNoSys_Handshapes.pdf. Dated 10 June 2010; drawings by Heiko Zienert, Olga Jeziorski, and Andreas Hanß
2010
-
[20]
Ferlin, M.et al.Quantifying inconsistencies in the hamburg sign language notation sys- tem.Expert Systems with Applications256, 124911 (2024)
2024
-
[21]
& Eccarius, P
Brentari, D. & Eccarius, P. Handshape contrasts in sign language phonology. In Brentari, D. (ed.)Sign Languages, Cam- bridge Language Surveys, 284–311 (Cam- bridge University Press, Cambridge, UK, 2010)
2010
-
[22]
Brentari, D., Coppola, M., Cho, P. W. & Senghas, A. Handshape complexity as a pre- cursor to phonology: Variation, emergence, and acquisition.Language Acquisition24, 283–306 (2017)
2017
-
[23]
& Bowden, R
Koller, O., Ney, H. & Bowden, R. Deep hand: How to train a cnn on 1 million hand images when your data is continuous and weakly labelled. In2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3793–3802 (IEEE, 2016)
2016
-
[24]
& Duh, K
Zhang, X. & Duh, K. Handshape-aware sign language recognition: Extended datasets and exploration of handshape-inclusive methods. InFindings of the Association for Computa- tional Linguistics: EMNLP 2023, 2993–3002 (Association for Computational Linguistics, Singapore, 2023). U...
2023
-
[25]
Lobo-Neto, V. C. & Pedrini, H. LSWH100: A handshapedatasetforbraziliansignlanguage (Libras)usingSignWriting.Data in Brief56, 110780 (2024)
2024
-
[26]
Sign language mnist
DataMunge. Sign language mnist. Kaggle dataset (2017). URL https://www.kaggle.c om/datasets/datamunge/sign-language-m nist. CC0: Public Domain; accessed 21 July 2026
2017
-
[27]
& Bowden, R
Pugeault, N. & Bowden, R. Spelling it out: Real-time ASL fingerspelling recogni- tion. In2011 IEEE International Confer- ence on Computer Vision Workshops (ICCV Workshops), 1114–1119 (IEEE, Barcelona, Spain, 2011)
2011
-
[28]
& Kwolek, B
Hosoe, H., Sako, S. & Kwolek, B. Recogni- tion of jsl finger spelling using convolutional neural networks. In2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA), 85–88 (IEEE, Nagoya, Japan, 2017)
2017
-
[29]
ADDSL: Hand gesture detec- tion and sign language recognition on anno- tated danish sign language.arXiv preprint arXiv:2305.09736(2023)
Jain, S. ADDSL: Hand gesture detec- tion and sign language recognition on anno- tated danish sign language.arXiv preprint arXiv:2305.09736(2023). URL https://arxi v.org/abs/2305.09736
2023 arXiv
-
[30]
A., Lanzarini, L
Ronchetti, F., Quiroga, F., Estrebou, C. A., Lanzarini, L. C. & Rosete, A. LSA64: An argentinian sign language dataset. InPro- ceedings of the XXII Congreso Argentino de Ciencias de la Computación (CACIC 2016), 794–803 (Red de Universidades con Carreras en Informática (RedUNCI...
2016
-
[31]
& Shishikui, Y
Kajiyama, T., Endo, R., Kaneko, H., Sano, M. & Shishikui, Y. Sign language image dataset with a hand pose type attribute. In2022 IEEE International Symposium on Broadband Multimedia Systems and Broad- casting (BMSB), 1–4 (IEEE, 2022)
2022
-
[32]
& Jahn, E
Hanke, T., Schulder, M., Konrad, R. & Jahn, E. Extending the public DGS cor- pus in size and depth. InProceedings of the LREC2020 9th Workshop on the Repre- sentation and Processing of Sign Languages: Sign Language Resources in the Service of the Language Community, Technologi...
2020
-
[33]
URL http://www.sign-lan g.uni-hamburg.de/glex
Konrad, R.et al.(eds.)Fachgebärdenlexikon Gesundheit und Pflege(Signum, Seedorf, Germany, 2007). URL http://www.sign-lan g.uni-hamburg.de/glex
2007
-
[34]
Matthes, S.et al.DICTA-SIGN—building a multilingual sign language corpus. In Proceedings of the LREC2012 5th Work- shop on the Representation and Processing of Sign Languages: Interactions between Cor- pus and Lexicon, 117–122 (European Lan- guage Resources Association (ELRA),...
2012
-
[35]
& Rutkowski, P
Łacheta, J., Czajkowska-Kisil, M., Linde-Usiekniewicz, J. & Rutkowski, P. (eds.)Korpusowy Słownik Polskiego Języka Migowego/Corpus-based Dictio- nary of Polish Sign Language(Faculty of Polish Studies, University of War- saw, Warsaw, Poland, 2016). URL https://www.slownikpjm.uw...
2016
-
[36]
& San-Segundo, R
Villa-Monedero, M., Gil-Martín, M., Sáez- Trigueros, D., Pomirski, A. & San-Segundo, R. Sign language dataset for automatic motiongeneration.Journal of Imaging9,262 (2023)
2023
-
[37]
B., Sinha, M
Varanasi, A. B., Sinha, M. & Dasgupta, T. Cross-linguistic phonological similarity anal- ysis in sign languages using HamNoSys. In Proceedings of the Workshop on Sign Lan- guage Processing (WSLP), 51–66 (Associa- tion for Computational Linguistics, IIT Bom- bay, Mumbai, India,...
2025
-
[38]
& Acharya, T
Mitra, S. & Acharya, T. Gesture recognition: A survey.IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews)37, 311–324 (2007). 18
2007
-
[39]
& Sun, J
He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770–778 (IEEE, 2016)
2016
-
[40]
InInternational Conference on Learning Representations(2021)
Dosovitskiy, A.et al.An image is worth 16x16 words: Transformers for image recogni- tion at scale. InInternational Conference on Learning Representations(2021). URLhttps: //openreview.net/forum?id=YicbFdNTTy
2021
-
[41]
URL https://arxi v.org/abs/2006.10214
Zhang, F.et al.MediaPipe Hands: On-device real-time hand tracking.arXiv preprint arXiv:2006.10214(2020). URL https://arxi v.org/abs/2006.10214
2020 arXiv
-
[42]
Kipf, T. N. & Welling, M. Semi-supervised classification with graph convolutional net- works. InInternational Conference on Learn- ing Representations(2017). URL https://op enreview.net/forum?id=SJU4ayYgl
2017
-
[43]
& Das, A
Sarkar, U., Chakraborti, A., Samanta, T., Pal, S. & Das, A. Enhancing asl recognition with gcns and successive residual connec- tions. In Palaiahnakote, S.et al.(eds.) Pattern Recognition. ICPR 2024 Interna- tional Workshops and Challenges, vol. 15616 ofLecture Notes in Comput...
2024
-
[44]
& Guestrin, C
Chen, T. & Guestrin, C. XGBoost: A scal- able tree boosting system. InProceedings of the 22nd ACM SIGKDD International Con- ference on Knowledge Discovery and Data Mining, 785–794 (Association for Computing Machinery, New York, NY, USA, 2016)
2016
-
[45]
& Aly, W
Aly, S. & Aly, W. DeepArSLR: A novel signer-independent deep learning framework for isolated arabic sign language gestures recognition.IEEE Access8, 83199–83212 (2020). 19
2020
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.