Expanding the Lexicon of Ge'ez Based African Languages: A Comparative Study of Amharic and Tigrinya

Debela Desalegn Yadeta; Hailay Kidu Teklehaymanot; Wolfgang Nejdl

REVIEW 4 major objections 6 minor 3 references

Expanding the Lexicon of Ge'ez Based African Languages: A Comparative Study of Amharic and Tigrinya

T0 review · 4 major / 6 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Extending XLM-R with 30,000 Ge'ez-script subwords and continued masked-language-model training raises question-answering exact match from 0.66 to 0.87 for Amharic and Tigrinya, and improves OOV-token NER accuracy from 81.4% to 94.3% on aver

desk verdict Sensible vocabulary-expansion idea and plausible headline numbers, but the ablation table contradicts its own text and the method is described two different ways — needs major revision before it can be trusted. read the letter →

arxiv 2607.15209 v1 pith:65CXRKWS submitted 2026-07-16 cs.CL

Hailay Kidu Teklehaymanot , Debela Desalegn Yadeta , Wolfgang Nejdl This is my paper · ORCID

classification cs.CL

keywords Ge'ezscriptAmharicTigrinyavocabularyexpansionsubwordtokenizationmultilinguallanguagemodelslow-resourceAfricanlanguagescontinuedpretraining

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the performance gap of multilingual language models on Ge'ez-script languages can be substantially closed by a targeted, low-cost adaptation: expanding XLM-R's vocabulary with 30,000 script-specific subwords and continuing masked-language-model training on Amharic and Tigrinya text before task fine-tuning. On question answering this raises exact match from 0.66 to 0.87 and F1 from 0.78 to 0.90; on sentiment analysis accuracy rises from 0.77 to 0.80; and on named-entity recognition, accuracy on out-of-vocabulary tokens increases from 81.4% to 94.3% on average across 11 African languages. The authors argue the gains carry over to 17 other African languages that received no vocabulary expansion, because continued pretraining strengthens general morphological representations. A sympathetic reader would care because the approach is cheap, backward-compatible with the original model, and offers a template for other underserved writing systems.

What carries the argument

The central object is the extended tokenizer and embedding matrix: a SentencePiece tokenizer trained on curated Amharic and Tigrinya corpora contributes 30,000 new subword tokens (50K units for Tigrinya, 32K for Amharic, deduplicated against the original 250K), and each new embedding is initialized as the mean of all source embeddings to keep the new tokens aligned with the existing representation space. The second moving part is two-stage training: continued masked language modeling over the extended vocabulary, which the ablations identify as the largest driver of gains, followed by task-specific fine-tuning that updates only the head and embedding layer. The mechanism as a whole mitigates

What would settle it

Inspect the released code's embedding initialization. If the new embeddings are set to the mean of all original 250K embeddings, Eq. (2) is what ran; if they are set to the average of constituent subword embeddings, the paper's stated method is not the one that produced the numbers, and the Table 5 ablation labeled 'Mean Init' does not test what it claims.

Watch

Extended reading notes

Core claim

VEXMLM is an XLM-R variant whose original 250K-token vocabulary is extended with 30,000 Ge'ez-script subwords produced by language-specific SentencePiece tokenizers for Amharic and Tigrinya. New token embeddings are initialized to the centroid of the source embedding space (Eq. 2), then the whole model is trained with the masked-language-model objective on monolingual corpora before being fine-tuned on QA, NER, and sentiment tasks. The reported results: 87.0 EM / 90.0 F1 on Amharic/Tigrinya QA vs. 66.0 / 78.0 for the base XLM-R; 80.0% sentiment accuracy vs. 77.0%; and average OOV-token NER accuracy rising from 81.4% to 94.3% across 11 languages. Ablations on Tigrinya NER indicate that contin

Load-bearing premise

The whole performance story rests on the training pipeline being exactly as described, but the paper gives two different recipes for initializing the new embeddings: the abstract says each token's embedding is the average of its constituent subwords' embeddings, while Section 3.3 Eq. (2) says every new token gets the mean of all 250K source embeddings; the reported results cannot be attributed to a uniquely specified method until that is resolved.

Editorial extensions

If this is right

QA exact match on Amharic/Tigrinya rises from 0.66 to 0.87 and F1 from 0.78 to 0.90, a 21-point EM gain over the unmodified base model.
Sentiment accuracy improves from 0.77 to 0.80, and OOV-token NER accuracy from 81.4% to 94.3% on average across 11 languages.
The 30,000-token expansion preserves all original vocabulary entries, so existing languages suffer no regression in coverage.
Ablations show continued MLM pretraining is the largest single driver of the OOV gains, with mean initialization helping convergence relative to random initialization.
The design is parameter-efficient: vocabulary grows from 250K to 280K and parameters from 279M to 301M.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper's own ablations imply that continued MLM training – not the script-specific vocabulary itself – is responsible for most of the cross-lingual gains; a natural next experiment would isolate vocabulary expansion from continued pretraining on a non-Ge'ez language to confirm that attribution.
Because the 30K-token expansion is script-agnostic in spirit, the same recipe should transfer to other high-OOV abugida or syllabary scripts (e.g., Devanagari, Bengali, Thai), where Latin-centric tokenizers cause similar fragmentation.
The unresolved initialization ambiguity (Eq. 2 vs. constituent averaging) means the reported mean-init advantage could be an artifact of the implementation; re-running the ablation with both initializations on the released code would settle which mechanism actually helps convergence.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know this paper has a solid core idea but is not ready for publication. The authors extend XLM-R with 30k Ge'ez-script subwords for Amharic and Tigrinya, then continue MLM pretraining and fine-tune on QA, NER, SA. They report big QA gains (EM 0.87 vs 0.66) and a 12.9-point average OOV accuracy improvement on NER. The approach is a combination of known techniques — vocabulary expansion, WECHSEL-style embedding initialization, continued pretraining — applied to a new script family, and that combination is sensible and worth testing.

What's good: the evaluation spans 19 languages, includes intrinsic tokenization metrics, and the comparison with Glot500 is informative. The authors also acknowledge that their vocabulary expansion only directly helps Ge'ez-script languages and that cross-lingual gains likely come from continued pretraining. That honesty is a point in their favor.

But there are serious problems. The ablation table (Table 5) reports XLM-R baseline 96.1, +vocab random 97.3, +mean 97.8, +continued pretraining 98.2. The text in §5.3 claims random init gives +1.2, mean init +2.3 over random, and continued pretraining +7.1. The actual increments are +1.2, +0.5, and +0.4. The +7.1 claim is arithmetically incompatible with the table. This is the only isolation of the continued-pretraining contribution, so the mechanism story — that continued pretraining drives the gains — is unsupported by the reported data. Second, the abstract describes initialization as averaging the embeddings of constituent subwords, while Eq. (2) defines it as the mean of all source embeddings. Those are different procedures. The downstream claims depend on which was actually used. Third, Table 7, meant to report NER macro-F1, is empty ('Full results pending model outputs'). And there's a reviewer note left in the conclusion of the manuscript ('One accuracy point worth double-checking before you finalize...'). These are not cosmetic problems; they mean the paper as submitted does not support its own claims.

The QA and SA headline numbers could still be real. The issues are fixable. But the current manuscript is not reliable, and a referee would have to do the authors' clean-up work. The evaluation on TIGQA, a dataset authored by the first author, is a minor concern but not disqualifying given it's published.

If this comes across your desk, send it back with a clear request: reconcile the ablation text with the table, specify the initialization procedure and verify it against the released code, fill in Table 7, and strip out the reviewer note. Then it could be a decent empirical contribution for the multilingual NLP community. As is, I would not trust the component attributions, but the core question is worth engaging with. I'd accept it for peer review only with the expectation of major revision.

Referee Report

4 major / 6 minor

Summary. The paper introduces VEXMLM, an XLM-R variant whose vocabulary is extended with 30,000 SentencePiece subword tokens trained on Amharic and Tigrinya. New embeddings are claimed to be initialized by averaging source embeddings; the model is then trained with continued masked language modeling and fine-tuned on QA, NER, and sentiment analysis. The authors report large gains over XLM-R on Amharic/Tigrinya QA (EM 0.66→0.87), sentiment accuracy (0.77→0.80), and average NER OOV-token accuracy (81.4%→94.3% across 11 languages), and they claim transfer to 19 African languages. An ablation on Tigrinya NER is intended to attribute the gains to vocabulary expansion, mean initialization, and continued pretraining.

Significance. If the reported results are reproducible, the paper addresses a genuine problem: Latin-script-centric tokenizers fracture Ge'ez-script languages, and targeted vocabulary extension plus continued pretraining is a practical and inexpensive fix. The public code/model/demo links are a strength, and the evaluation spans a useful set of African languages and tasks. However, the manuscript currently contains a direct numerical inconsistency in the central ablation, a conflict between the abstract and Eq. (2) for embedding initialization, and an empty appendix table. These issues prevent the reader from verifying the paper's main attribution claim and must be fixed before the contribution can be accepted.

major comments (4)

[§5.3, Table 5] The ablation text and Table 5 are arithmetically inconsistent. The table rows (96.1, 97.3, 97.8, 98.2) imply increments of +1.2, +0.5, and +0.4 for random init, mean init, and continued pretraining, respectively. The text, however, claims mean init gives +2.3 over random init and continued pretraining gives the largest single gain of +7.1. The +7.1 value is not derivable from any row pair in this table; it would require a baseline near 91.1, which is the Amharic OOV number in Table 3, not the Tigrinya baseline. Because Table 5 is the only direct evidence isolating component contributions, the central claim that continued pretraining is the most important component is currently unsupported. Please correct the text or table and report the actual component-wise increments.
[Abstract; §3.3, Eq. (2)] The description of embedding initialization is internally inconsistent. The abstract states that new embeddings are initialized by averaging the embeddings of constituent subwords under XLM-R's original tokenizer, which is the WECHSEL-style constituent averaging. In contrast, Eq. (2) defines each new embedding as the mean of all source embeddings in V_s. These are different procedures, and the behavior of the released model depends on which one was implemented. This ambiguity affects both reproducibility and the interpretation of the ablation. Please specify the exact algorithm and align the prose, equations, and code.
[Appendix B, Table 7] The appendix promises macro-F1 scores for NER 'alongside accuracy,' but Table 7 contains only a header and the text 'Full results pending model outputs.' This is an incomplete result section, not a finished evaluation. Since §7 explicitly says accuracy should be considered together with macro-F1, the omission is load-bearing for the NER claims. The table must be populated before the manuscript can be considered complete.
[§5.4/§7 and note following §6] The paper's 19-language framing conflicts with the fact that vocabulary augmentation targets only Ge'ez-script Amharic and Tigrinya. The largest OOV gains in Table 3 occur in non-Ge'ez languages (e.g., Swahili +15.0, Kinyarwanda +8.5), while the target languages gain only +1.1 and +2.1. Section 5.4 and §7 acknowledge these gains are a by-product of continued pretraining, not of the vocabulary component. The unedited note following §6 also tells the authors to clarify exactly this point. As written, the paper presents a 19-language average as evidence for a Ge'ez-specific intervention. Please report Ge'ez-script and non-Ge'ez-script results separately and align the claims with the acknowledged mechanism.

minor comments (6)

[Table 3] The caption says 'not directly comparable to the overall NER accuracy reported in Table 3,' but the table itself is numbered Table 3. The reference should point to the downstream NER table (e.g., Table 4 or the appendix macro-F1 table).
[§5.1, Table 2] The parity interpretation is inconsistent. The text says values closer to 1.0 indicate more equitable tokenization, but then calls VEXMLM's very low Tigrinya parity (0.27) 'desirable' because it is compact. If compactness is the goal, the metric should be defined and interpreted as efficiency, not parity/equity.
[§4.5, Table 5] The last row of Table 5 is labeled '+ Continued Pretraining,' while §4.5 defines the fourth configuration as 'VEXMLM (Full): Mean init + continued pretraining + fine-tuning.' Clarify whether the last row includes all previous components or only continued pretraining on top of the mean-initialized model.
[Appendix A, Table 6] The language table lists 19 languages, but the NER OOV analysis in Table 3 covers only 11 and the appendix macro-F1 table is empty. Please make clear, for each task, how many languages are actually evaluated and why the remaining languages are excluded.
[References] Some references are malformed. For example, the XLM-R entry appears as 'Kartikay Conneau, Alexis Workshop Khandelwal...', and the Wang et al. (2019) entry contains duplicated text. Please clean up the reference list.
[After §6] The unedited internal note beginning 'One accuracy point worth double-checking before you finalize...' should be removed. It is not scholarly prose and should not appear in a submission.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the empirical pipeline is evaluated against external benchmarks, and the self-cited QA dataset is independent published data; the ablation arithmetic inconsistency is a correctness issue, not circularity.

full rationale

The paper does not derive its downstream results from its own equations or fitted parameters. Vocabulary expansion (Section 3.2), embedding initialization (Section 3.3, Eq. 2), and two-stage training (Section 3.4) are described as construction choices, and the reported QA/SA/NER numbers are measured against external published benchmarks (AfriSenti, MasakhaNER, TIGQA, AmQA). The single self-citation, TIGQA (Teklehaymanot et al., 2024), is a published LREC-COLING dataset by the first author; it is externally falsifiable and not used to justify the method, so under the review rules it does not raise the circularity score. No prediction is a renamed fit; no uniqueness theorem or prior self-citation is invoked to force the architecture. The most serious manuscript problem is an internal ablation inconsistency: Section 5.3 claims random initialization gives +1.2, mean initialization +2.3 over random, and continued pretraining +7.1, while Table 5's rows (96.1 -> 97.3 -> 97.8 -> 98.2) imply increments of +1.2, +0.5, and +0.4. This makes the component-level attribution unsupported, but it is an arithmetic/reproducibility flaw, not a circular reduction. The appended editorial note and Section 7 limitations likewise acknowledge that non-Ge'ez-script gains come from continued pretraining rather than the Ge'ez vocabulary; this is a scope/attribution caveat, not circularity.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The method is an empirical engineering recipe; it introduces no new theoretical entities. Its results rest on tokenizer vocabulary sizes, training sequence truncation, and the untested premise that continued pretraining on two languages transfers to seventeen others. The model VEXMLM itself is a configuration, not a new physical or mathematical entity.

free parameters (4)

Amharic SentencePiece vocabulary size = 32,000
Chosen to minimize OOV on held-out evaluation corpora (Section 3.2).
Tigrinya SentencePiece vocabulary size = 50,000
Chosen to balance script coverage with computational efficiency (Section 3.2).
Sequence length limit = 256 tokens
Imposed for computational efficiency; the paper acknowledges it may constrain QA on long documents (Section 7).
Number of new vocabulary tokens = 30,000
Result of deduplicating the new tokenizers against XLM-R's vocabulary; not independently tuned but treated as a fixed design choice.

assumptions (3)

ad hoc to paper Continued MLM pretraining on Amharic and Tigrinya corpora transfers to 17 other African languages without training data from those languages.
Central to contribution (ii); Section 5.4 admits the non-Ge'ez gains come from continued pretraining, but no mechanism or training data for those languages is provided, making this an assumed transfer effect.
domain assumption Mean-based embedding initialization positions new tokens in a way that avoids destabilizing pretraining.
Section 3.3 asserts this without a theoretical guarantee; the ablation shows only a small gain on a single language, so the generality of the assumption is unverified.
domain assumption Keeping all original XLM-R vocabulary entries preserves backward compatibility and avoids regression on the 100 original languages.
Stated in Section 3.2; plausible but not empirically verified for all tasks and languages.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Expanding the Lexicon of Ge'ez Based African Languages: A Comparative Study of Amharic and Tigrinya." pith.science (2026). https://pith.science/paper/65CXRKWS

@misc{pith2026260715209,
  author       = {Pith},
  title        = {Pith review of: Expanding the Lexicon of Ge'ez Based African Languages: A Comparative Study of Amharic and Tigrinya},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/65CXRKWS}},
  note         = {Machine review of arXiv:2607.15209}
}

read the original abstract

Multilingual pre-trained language models (PLMs) exhibit degraded performance on low-resource, non-Latin-script languages, driven by high out-of-vocabulary (OOV) rates and excessive subword fragmentation that result from Latin-script-centric tokenizer training. We introduce VEXMLM, a vocabulary-extended variant of XLM-R targeting the two highest-resource Ge'ez-script languages, Amharic and Tigrinya, and further evaluated on 17 additional low-resource African languages (19 total). We train a language-specific SentencePiece tokenizer on curated Amharic and Tigrinya monolingual corpora, extend XLM-R's vocabulary with 30,000 Ge'ez-script subwords derived from this tokenizer, and initialize their embeddings by averaging the embeddings of their constituent subwords under XLM-R's original tokenizer. VEXMLM is trained in two stages: (1) continued masked language modeling over the extended vocabulary on the curated corpora, and (2) supervised fine-tuning on question answering (QA), named entity recognition (NER), and sentiment analysis (SA). On Amharic/Tigrinya QA, VEXMLM achieves 87.0 EM /90.0 F1, versus 66.0 EM/78.0 F1 for XLM-R and 74.0 EM/ 78.0 F1 for Glot500. On SA, VEXMLM reaches 80.0\% accuracy versus 77.0\% (XLM-R) and 46.0\% (Glot500). On NER, VEXMLM raises OOV-token entity accuracy from 81.4\% to 94.3\%, averaged over 11 of the 19 evaluated languages for which OOV analysis was possible. Our contributions are: (i) a vocabulary-extension and embedding-initialization procedure tailored to Ge'ez script; (ii) a two-stage training strategy under which vocabulary and continued-pretraining gains on Amharic/Tigrinya transfer to 17 typologically related, unaugmented African languages; and (iii) an evaluation spanning both intrinsic tokenization metrics (vocabulary coverage, fertility, OOV rate) and extrinsic task performance across all 19 languages.

Figures

Figures reproduced from arXiv: 2607.15209 by the authors.

**Figure 2.** Vocabulary size comparison across languages for XLM-R (250K) and VEXMLM (280K). VEXMLM’s expanded vocabulary is uniformly available across all evaluated languages. OOV Handling [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

3 extracted references · 3 linked inside Pith

[2]

arXiv preprint arXiv:2402.14714

Efficient and effective vocabulary expansion towards multilingual large language models. arXiv preprint arXiv:2402.14714. Taku Kudo and John Richardson. 2018. SentencePiece: A simple and language independent subword tok- enizer and detokenizer for neural text processing . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Proce...

arXiv 2018
[2023]

In Findings of the Association for Computational Linguistics: ACL 2023 , pages 5661–5681, Toronto, Canada

Tokenization impacts multilingual language modeling: Assessing vocabulary allocation and over- lap across languages. In Findings of the Association for Computational Linguistics: ACL 2023 , pages 5661–5681, Toronto, Canada. Association for Com- putational Linguistics. Yihong Liu, Peiqin Lin, Mingyang Wang, and Hinrich Schuetze. 2024. OFA: A framework of i...

arXiv 2023
[2024]

In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 1582–1599, Miami, Florida, USA

Tokenization falling short: On subword ro- bustness in large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 1582–1599, Miami, Florida, USA. As- sociation for Computational Linguistics. Ethan C. Chau and Noah A. Smith. 2021. Specializing multilingual language models: An empirical study . In Proceedings of t...

arXiv 2024

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.

[1] [2]

arXiv preprint arXiv:2402.14714

Efficient and effective vocabulary expansion towards multilingual large language models. arXiv preprint arXiv:2402.14714. Taku Kudo and John Richardson. 2018. SentencePiece: A simple and language independent subword tok- enizer and detokenizer for neural text processing . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Proce...

arXiv 2018

[2] [2023]

In Findings of the Association for Computational Linguistics: ACL 2023 , pages 5661–5681, Toronto, Canada

Tokenization impacts multilingual language modeling: Assessing vocabulary allocation and over- lap across languages. In Findings of the Association for Computational Linguistics: ACL 2023 , pages 5661–5681, Toronto, Canada. Association for Com- putational Linguistics. Yihong Liu, Peiqin Lin, Mingyang Wang, and Hinrich Schuetze. 2024. OFA: A framework of i...

arXiv 2023

[3] [2024]

In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 1582–1599, Miami, Florida, USA

Tokenization falling short: On subword ro- bustness in large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 1582–1599, Miami, Florida, USA. As- sociation for Computational Linguistics. Ethan C. Chau and Noah A. Smith. 2021. Specializing multilingual language models: An empirical study . In Proceedings of t...

arXiv 2024