{"id":"cbe0c3f9-54cd-4a29-89fb-2805a2548d00","arxiv_id":"2505.24355","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A multilingual gloss-free sign language translation model with token-level language identification and text CTC alignment gets competitive BLEU and ROUGE on SP-10, PHOENIX14T, and CSL-Daily.","lead":"A new model translates videos of ten sign languages directly into written words without using gloss annotations, using two alignment objectives to cope with multiple languages at once. The method posts competitive scores on three sign language benchmarks and is a step toward a single model for many sign languages.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The LIDtok target repeats one language ID |T| times, so Eq. 2 supplies a word-count prior rather than a token-level sign-to-word alignment, and the many-to-one ablation isolating LIDtok is missing.","rationale":"The reader's weakest assumption points to the same mechanism: the LIDtok CTC objective assumes an alignment between encoder outputs and a token-length LID sequence. My reading sharpens this into a more specific technical problem: because the LIDtok target is the same ID repeated |T| times, CTC training can only encode the spoken word count, not any token-level correspondence. This directly affects the central claim that dual CTC objectives handle language conflicts through token-level SLI. The missing many-to-one ablation is also load-bearing because many-to-one is the setting where language conflicts are explicitly measured and where the model claims to surpass individual models. The paper is otherwise honest about limitations and provides useful first benchmarks, so the appropriate verdict remains CONDITIONAL rather than acceptance or rejection: the architecture is plausible and the results are suggestive, but the central LIDtok mechanism needs an alignment-quality check and an ablation in the primary language-conflict setting before the claim is secure.","tokens_in":10972,"tokens_out":4845,"duration_ms":69483,"concrete_test":"Run a many-to-one ablation on SP-10 (10 SLs to English) with three variants: (a) the full method, (b) the full method with LIDtok replaced by a single utterance-level LID objective, and (c) the full method with LIDtok target always a single LID token instead of |T| repetitions. If (b) or (c) matches (a) within noise, the token-length aspect of Eq. 2 is not load-bearing. Additionally, extract argmax CTC alignments from the initial encoder for a sample of SP-10 dev videos and check whether non-blank LID emissions are distributed across the sign sequence or collapse to a small number of frames; if the latter, the objective is not performing token-level alignment.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central novelty is token-level sign language identification, but Eq. 2 does not actually provide token-level information. The target Itok is defined as the same SL ID repeated |T| times, where |T| is the length of the spoken translation. Under CTC, identical adjacent labels must be separated by blanks, so the only supervision carried by this objective is the count of spoken words, not which sign or sign unit corresponds to which word. Sign languages do not have explicit word boundaries, and their constituent order often differs from the target spoken language; there is no principled reason that the number of sign-language units should equal |T|. The phrase 'explicitly aligns the sign representations with each spoken word' in Section 3.2 is therefore not supported by the construction. This matters because Table 9 is the only direct evidence for LIDtok, and it shows that LIDtok actually hurts at the 2-pair and 3-pair settings (7.48 vs 6.22 and 6.50 vs 6.15) before helping at larger pair counts. The many-to-one setting, where language conflicts are explicitly quantified (Table 8, mean drop of 1.50 BLEU), has no with/without-LIDtok ablation: Table 4 compares the full Sign2(LID+Text) against a universal baseline, so the gains could come from TxtCTC alone, which is already shown to help in one-to-one settings. Without an isolation of LIDtok in many-to-one and without evidence that the CTC path actually emits LID tokens at meaningful positions, the paper's central claim that LIDtok resolves language conflicts remains unverified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Sign2(LID+Text), a multilingual gloss-free sign language translation model with a hierarchical encoder trained with two auxiliary CTC objectives: a token-level sign language identification (LIDtok) objective applied at an early encoder layer and a text CTC objective applied at the final encoder layer. The model is evaluated in one-to-one, many-to-one, and many-to-many settings on SP-10, PHOENIX14T, and CSL-Daily, and the authors report competitive BLEU/ROUGE scores, including a many-to-one improvement over a universal baseline and a new one-to-one benchmark on SP-10. The paper also releases code and model weights.","tokens_in":11271,"tokens_out":4029,"duration_ms":50494,"significance":"If the empirical claims held, the paper would be a useful step toward a multilingual gloss-free SLT model, and the released code plus the first one-to-one SP-10 benchmark would be valuable community resources. The many-to-one comparison against MLSLT and the incremental many-to-many experiments address an underexplored problem. However, the central novelty—token-level sign language identification—is not well supported by the construction in Eq. (2), and several empirical claims are overstated relative to the tables, so the significance depends on substantial revision.","major_comments":[{"comment":"The LIDtok target Itok is defined as the same language ID repeated |T| times, where |T| is the length of the spoken text. Under CTC, repeated identical labels must be separated by blanks, so this objective can only supervise the number of output positions, not align specific sign representations to specific spoken words. The statement in §3.2 that the |T|-length LIDtok sequence 'explicitly aligns the sign representations with each spoken word' is therefore not supported by the construction. The paper should either provide evidence that the CTC path emits LID tokens at meaningful positions (e.g., analysis of intermediate predictions or attention maps) or reframe the contribution as a length-prior/word-count objective rather than token-level identification.","section":"§3.2, Eq. (2)"},{"comment":"The many-to-one section attributes the improvement over the universal baseline to mitigation of language conflicts, but Table 4 compares the full Sign2(LID+Text) model only against an individual/universal baseline and prior methods. There is no ablation that removes LIDtok while keeping TxtCTC, even though one-to-one experiments already show that TxtCTC alone yields sizable gains. Without this ablation, the improvement cannot be attributed to Sign2LID, and the conclusion in §6 that 'Sign2LID effectively mitigates language conflicts' is unsupported.","section":"§5, many-to-one, Table 4"},{"comment":"The ablation study in Table 9 shows that LIDtok hurts performance at the 2-pair and 3-pair settings (7.48 vs. 6.22 and 6.50 vs. 6.15 BLEU, respectively) before helping at 4 pairs and beyond. Since the paper reports no error bars or significance tests on a very small dataset (830 training samples per language), these differences may be within noise. The claim that 'LIDtok is especially effective under more challenging translation conditions' requires a statistical basis and an explanation of the harmful regime at small pair counts.","section":"Appendix E, Table 9"},{"comment":"The introduction claims the model 'achieve[s] state-of-the-art performance for each task,' but Table 3 shows that on CSL-Daily the proposed model (BLEU 14.18) is below SignLLM (BLEU 15.75), and similar discrepancies exist in ROUGE. The abstract more cautiously says 'competitive performance,' but the contribution statement is not consistent with the reported results. The claim should be revised to match the tables, or the comparison should be explained (e.g., different training data or decoding settings).","section":"§1, contribution statement vs. Table 3"}],"minor_comments":[{"comment":"In the Dev-ROUGE block, the mean value for 'Individual (10)' is reported as 28.53, but the individual ROUGE values listed range from roughly 27.70 to 36.60, whose arithmetic mean is about 32.16. This appears to be a typographical error and should be corrected.","section":"Table 4"},{"comment":"The text says the many-to-many model 'maintains comparable performance as the number of language pairs increases to five,' but the BLEU gap between one-to-one and many-to-many grows from 0.06 at 2 pairs to 0.97 at 5 pairs. The wording should be softened or the relevant pairwise significances reported.","section":"Table 5"},{"comment":"Please report whether the reported BLEU scores are averaged over multiple random seeds; for a dataset as small as SP-10, single-run results are not sufficient to judge the direction of the ablation.","section":"Appendix E / Table 9"},{"comment":"The model name is inconsistently rendered as 'Sign2(LID+Text)' in the abstract and main text but 'Sign2LID' in §3.2 and 'Sign2 (LID+Text)' in Table 1. Please standardize the notation.","section":"Notation"},{"comment":"The paper cites 'Higuchi et al. (2022)' for hierarchical conditioning and 'Yan et al. (2023)' for joint CTC/attention decoding; both are appropriate, but the text would benefit from a brief sentence clarifying how these prior works were adapted to the sign-language setting, particularly the choice of the initial encoder layer for LIDtok.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper's central claim about token-level SL identification is not yet supported by the construction or the ablations, and the introduction's state-of-the-art claim conflicts with its own Table 3. These issues are fixable in revision: reframe the LIDtok contribution, add a many-to-one LIDtok ablation, add error bars or multiple runs, and correct the contribution statement. The released code and the first one-to-one SP-10 benchmark are genuine assets. I recommend major_revision rather than rejection because the empirical direction (multilingual gloss-free SLT with auxiliary CTC) is plausible and the identified gaps can be addressed within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The useful core here is the first gloss-free multilingual SLT benchmark: one-to-one results on nine SP-10 pairs, plus many-to-one and many-to-many comparisons, all with BLEU/ROUGE against reasonable baselines. The many-to-one gains over MLSLT (5.18 vs 4.66) are real, and the gap over the universal baseline (3.10) is stark. TxtCTC is a sensible borrow from ASR, and the length-based analysis in Figures 3–4 is informative.\n\nThe problem is the central novelty. The LIDtok target is the same SL ID repeated |T| times, where |T| is the spoken-text length. That supplies a word-count prior plus a language label, not the token-level alignment claimed in §3.2. The sentence about explicitly aligning sign representations with each spoken word is not supported by Eq. 2. The only direct ablation (Table 9) shows LIDtok hurts at 2 and 3 pairs (7.48→6.22, 6.50→6.15) before helping from 4 pairs onward. And there is no many-to-one ablation isolating LIDtok, so the claimed language-conflict resolution is unverified. The introduction's 'state-of-the-art for each task' also overreaches: on CSL-Daily test, SignLLM's 15.75 beats this model's 14.18.\n\nI'd send this to review because the empirical skeleton is useful, but I'd push the authors to either demonstrate actual token-level alignment (e.g., via CTC emissions or attention analysis) or reframe LIDtok as language-and-length supervision. They should also fix the SOTA claim and add the missing many-to-one ablation.","headline":"First gloss-free multilingual SLT benchmark, but the LIDtok novelty is a word-count prior, not an alignment.","tokens_in":11854,"tokens_out":3695,"would_cite":true,"duration_ms":45606,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single gloss-free model with dual CTC objectives translates ten sign languages across one-to-one, many-to-one, and many-to-many settings, while resolving language conflicts.","keywords":["sign language translation","multilingual","gloss-free","connectionist temporal classification","token-level language identification","many-to-many translation","SP-10","PHOENIX14T"],"falsifier":"A concrete falsifier would be to shuffle the token-level sign-language ID labels within each training utterance and retrain the many-to-one model: if test BLEU does not fall measurably, then LIDtok is not the mechanism resolving language conflict. A more direct check is to inspect the initial encoder's CTC alignment posteriors and test whether language-ID tokens switch at points matching visible sign boundaries; if the alignments are arbitrary, the auxiliary supervision is not doing the alignment work the paper attributes to it.","tokens_in":10710,"feed_emoji":"🧏","tokens_out":9243,"duration_ms":96047,"temperature":0.7,"pith_summary":"This paper aims to establish that one multilingual, gloss-free model can translate sign language videos from ten sign languages into their paired spoken languages under three different task configurations: one-to-one, many-to-one, and many-to-many. The proposed architecture, Sign2(LID+Text), adds two auxiliary CTC objectives to a Transformer encoder: a token-level sign language identification loss (LIDtok) on the initial encoder layer and a text CTC loss (TxtCTC) on the final encoder layer. The paper reports that this design mitigates the language conflicts that otherwise cause a 1.50 BLEU drop in a universal many-to-one baseline, and that it achieves competitive or leading results on SP-10, PHOENIX14T, and CSL-Daily. A sympathetic reader would care because gloss-free and multilingual are the two properties needed to make sign language translation practical at scale: no gloss annotations are required, and one model can serve many language pairs.","feed_headline":"One model translates 10 sign languages without glosses","feed_subtitle":"Two CTC objectives separate language identity from translation alignment, making multilingual sign language translation practical.","key_machinery":"The load-bearing mechanism is a hierarchical encoder supervised by two auxiliary CTC (Connectionist Temporal Classification) objectives. The initial encoder layer is supervised by a token-level sign language identification loss ($L_{\\mathrm{LID}}$) that predicts a sequence of language IDs whose length equals the number of words in the target spoken text. The final encoder layer is supervised by a text CTC loss ($L_{\\mathrm{Txt}}$) that aligns the latent sign representation with the spoken text, allowing the encoder to reorder non-monotonically while the attention decoder generates the translation with CTC as an auxiliary beam-search guide. The full objective is $L_{\\mathrm{total}} = \\lambda_1 L_{\\mathrm{LID}} + \\lambda_2 L_{\\mathrm{Txt}} + \\lambda_3 L_{\\mathrm{Attn}}$ with weights $1$, $5$, and $3$ in the experiments.","core_discovery":"The central claim is that token-level sign language identification is the missing mechanism for multilingual gloss-free translation. Instead of one utterance-level language label, the model predicts a sequence of language IDs with the same length as the target spoken text, so every word position receives a fine-grained language cue; a second text-oriented CTC objective then reorders the latent sign representations into the spoken sentence within a joint CTC/attention decoder. On the paper's own numbers, the resulting model beats the existing multilingual baseline on many-to-one SP-10 translation (5.18 test BLEU versus 4.66 for MLSLT), keeps many-to-many quality close to one-to-one quality up to five language pairs, and improves one-to-one test BLEU over its vanilla baseline on PHOENIX14T (24.23 versus 22.52) and CSL-Daily (14.18 versus 11.76). The paper also presents the first gloss-free one-to-one SLT results on SP-10 for all ten languages, thereby giving the field a common reference point for multilingual gloss-free work.","pith_inferences":["The token-level LID supervision may actually be learning sign subunits or articulatory segments rather than word-sized units; if so, the same objective could double as an unsupervised segmentation signal for sign language recognition.","Because the LID sequence length is tied to the spoken text length, the method could fail for sign languages whose word order diverges sharply from the spoken language; testing it on verb-final versus verb-initial sign language pairs would reveal where the monotonicity prior breaks.","The dual-CTC recipe mirrors successful multilingual speech recognition designs, so a plausible next step is to replace the hand-assigned LID labels with continuous language embeddings and train the text-CTC head on far larger multilingual corpora, which the paper does not itself attempt."],"forward_implications":["If the dual-CTC design is correct, one model can serve ten sign languages without gloss supervision, removing the need to annotate glosses for every new language.","The many-to-many results imply that cross-lingual transfer can help low-resource sign languages: translation quality stays close to one-to-one quality when up to five language pairs are trained together.","Token-level language identification is more helpful as the number of mixed languages grows, so the benefit should increase as the model scales toward a true sign language foundation model.","The paper's one-to-one SP-10 benchmark, the first gloss-free multilingual benchmark on that corpus, gives later work a fixed point of comparison across all ten languages.","Since TxtCTC helps most on short and medium-length sentences, the alignment gains should concentrate where frame-to-token correspondences are clearest rather than on long, globally-structured sentences."],"supporting_citations":[{"why":"Supplies the CTC objective used by both LIDtok and TxtCTC, the core alignment mechanism.","marker":"Graves et al. (2006)"},{"why":"Shows token-level language identification in multilingual ASR, the direct template for LIDtok.","marker":"Chen et al. (2023)"},{"why":"Contributes the multilingual SP-10 corpus and the MLSLT baseline that many-to-one results are compared against.","marker":"Yin et al. (2022)"},{"why":"Provides the PHOENIX14T dataset and the original end-to-end Sign2Text baseline.","marker":"Camgoz et al. (2018)"},{"why":"Establishes hierarchical multi-task CTC encoding, used as the encoder structure.","marker":"Sanabria and Metze (2018)"},{"why":"Demonstrates text-CTC alignment within joint CTC/attention decoding for translation, the basis of TxtCTC.","marker":"Yan et al. (2023)"},{"why":"Supplies the SlowFastSign pretraining used to build the sign feature extractor.","marker":"Ahn et al. (2024)"},{"why":"Offers the most recent many-to-one MLSLT baseline on SP-10 using language-family clustering.","marker":"Zhang et al. (2025)"}],"fun_headline_variants":["One model, 10 sign languages, zero glosses","Token-level language ID powers multilingual gloss-free SLT","Dual CTC makes 10 sign languages translatable in one pass","From 1:1 to 10:10: gloss-free sign translation scales up","Fine-grained language cues enable multilingual sign translation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the model can learn a reliable CTC alignment between the first encoder layer's output and a token-level sequence of sign-language IDs whose length equals the number of words in the spoken text, even though sign languages have no explicit word boundaries and may order words differently from the spoken language.","fun_headline_variants_meta":{"raw":{"variants":["One model, 10 sign languages, zero glosses","Token-level language ID powers multilingual gloss-free SLT","Dual CTC makes 10 sign languages translatable in one pass","From 1:1 to 10:10: gloss-free sign translation scales up","Fine-grained language cues enable multilingual sign translation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000389,"raw_usage":{"total_tokens":2033,"prompt_tokens":913,"completion_tokens":1120,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":529,"completion_tokens_details":{"reasoning_tokens":1035}},"tokens_in":529,"tokens_out":1120,"duration_ms":14012,"temperature":1.0,"reasoning_tokens":1035,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:25:11.738673+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete falsifier would be to shuffle the token-level sign-language ID labels within each training utterance and retrain the many-to-one model: if test BLEU does not fall measurably, then LIDtok is not the mechanism resolving language conflict. A more direct check is to inspect the initial encoder's CTC alignment posteriors and test whether language-ID tokens switch at points matching visible sign boundaries; if the alignments are arbitrary, the auxiliary supervision is not doing the alignment work the paper attributes to it.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the PHOENIX14T dataset and the original end-to-end Sign2Text baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Offers the most recent many-to-one MLSLT baseline on SP-10 using language-family clustering."}],"review_version":1}