Pith. sign in

REVIEW 3 major objections 5 minor 31 references

Revisiting Projection-based Data Transfer for Cross-Lingual Named Entity Recognition in Low-Resource Languages

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Pith's one-line read Projection-based NER, reformulated as alignment-scored candidate matching, rivals and sometimes beats multilingual models on 57 low-resource languages.

desk verdict A solid, honest empirical study of a formalized candidate-matching projection method for cross-lingual NER, but the abstract overclaims: on the low-resource MasakhaNER2 benchmark the proposed method does not beat the heuristic AWESoME baseline on average. read the letter →

arxiv 2501.18750 v1 pith:EURQQ6OL submitted 2025-01-30 cs.CL cs.IR

classification cs.CLcs.IR
keywords cross-lingualnamedentityrecognitionannotationprojectionlow-resourcelanguageswordalignmentbipartitematchingtranslate-testMasakhaNER2XTREME
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that data-based annotation projection, done carefully, is a competitive and cheaper alternative to fine-tuning multilingual language models for named entity recognition in low-resource languages. The core move is to replace heuristic fixes for misaligned projections with a formal matching between source entities and candidate target spans, scored by word-alignment counts. Across experiments on 57 languages, the paper finds this candidate-matching projection generally beats previous alignment-based projection methods and, when candidates are extracted by the same NER model used for transfer, can outscore model transfer on the MasakhaNER2 low-resource benchmarks. A second proposed idea—computing alignments between the original sentence and its back-translated labelled version in the same language—did not improve results. If the main claim holds, low-resource NER can be done with off-the-shelf translation and alignment tools plus a small projection optimiser, without retraining multilingual models.

What carries the argument

The carrying mechanism is the candidate-matching formulation: a constrained weighted bipartite matching between source entity spans $S$ and target candidate spans $T$, with the matching score $c_{p_{\text{src}},p_{\text{tgt}}} = a_{p_{\text{src}},p_{\text{tgt}}} / ((j_{\text{src}}-i_{\text{src}}) + (j_{\text{tgt}}-i_{\text{tgt}}))$, where $a$ is the number of aligned word pairs between a source entity and a target candidate. This score rewards many aligned words while penalising candidates with many unaligned words, which handles single-word misalignments. Target candidates are generated either as all n-grams or as spans predicted by a multilingual NER model with class labels ignored; the problem is solved with a greedy approximate algorithm, and in the NER-candidate case it reduces to a maximum-weight bipartite matching. The alignments feeding the scores come from SimAlign and AWESoME, and the paper also experiments with a same-language (target-to-target) alignment direction enabled by marker-based back-translation.

What would settle it

Take a set of manually word-aligned parallel sentences in a morphologically complex language such as Xhosa, run the proposed projection with perfect alignments and with SimAlign/AWESoME alignments, and compare the projected NER F1; a small gap would falsify the paper's claim that projection quality is bounded by alignment quality.

Watch

Extended reading notes

Core claim

The paper argues that the weak point of projection-based cross-lingual NER is not the translation or the source NER model but the projection step, where previous methods patched misalignments with heuristics. Its central claim is that treating projection as a constrained matching problem—generate candidate spans in the target sentence, score each source-entity/candidate pair by the number of aligned words normalised by both span lengths, and solve the weighted bipartite matching greedily—generalises the heuristic methods and, with NER-model-based candidate extraction, outperforms them and can beat direct multilingual model transfer on low-resource African languages. The paper also proposed aligning the original and back-translated sentence in the same language (tgt2tgt) to improve alignment quality, and reports that this direction did not help because back-translation errors dominated.

Load-bearing premise

The method assumes the word-to-word alignments between the translated source sentence and the original target sentence are accurate enough for the matching score in Equation 2 to identify the correct target span; the paper's own results on Xhosa and Zulu show this assumption breaks for morphologically complex low-resource languages.

Editorial extensions

If this is right

  • A projection pipeline using the proposed matching can serve as a zero-shot NER labelling service for low-resource languages, needing only a translation model, a word aligner, and a source-language NER model.
  • Because the ILP formulation accepts arbitrary scoring functions, the same projection step can later fuse alignment scores with translation-probability scores or model-based scores without reworking the pipeline.
  • Using the same NER model for candidate extraction as for model transfer lets projection correct mislabelled spans, so projected training data can exceed the quality of the multilingual model's own predictions on low-resource languages.
  • Languages with simpler morphology (Yoruba, Bambara, and others) benefit most, while morphologically complex languages such as Xhosa and Zulu still need adaptation, so the method's coverage across Africa is uneven.
  • The target-to-target alignment direction is not worth adopting as-is in translate-test pipelines, since back-translation errors erase the expected same-language alignment gains, with Japanese as the only exception.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The matching formulation is language-agnostic enough that it should transfer to other span-labelling tasks, such as semantic role labelling or time-expression tagging, whenever source annotations and translated target text are available; the paper does not test this.
  • Combining the alignment score from Equation 2 with a translation-based similarity score in a weighted sum, which the ILP's design explicitly permits, could rescue the morphologically complex languages where alignments alone fail.
  • The failure of the target-to-target direction suggests the bottleneck is back-translation noise rather than alignment direction; using marker-based back-translation with constrained decoding, so that entities survive translation exactly, might make the same-language alignment idea viable.
  • If an exact polynomial-time solver for the overlap-constrained matching problem exists, replacing the greedy algorithm would remove approximation error and make the contribution of the matching formulation easier to isolate; the paper leaves this as an open question.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper proposes two enhancements to projection-based cross-lingual NER: (i) computing word-to-word alignments between the original target sentence and its back-translated labelled counterpart (tgt2tgt alignment direction), and (ii) a formal candidate-matching formulation in which source entity spans are matched to target candidate spans by solving a constrained weighted bipartite matching problem with alignment-based scores. Candidate spans are extracted either as all n-grams or with a multilingual NER model. The methods are evaluated on XTREME (39 languages) and MasakhaNER2 (18 languages) in full translate-test pipelines using NLLB-200, XLM-R-Large, SimAlign and AWESoME, plus an isolated projection-step evaluation on Europarl. The paper claims that the candidate-matching approach surpasses previous heuristic alignment-based projection methods in low-resource settings and that projection-based data transfer is a robust alternative to model-based transfer.

Significance. The paper releases its code and evaluation results, formalizes the projection step as an ILP, and honestly reports the failure of the tgt2tgt alignment direction. If the empirical superiority claim were supported, the candidate-matching formulation would be a useful practical alternative for low-resource NER. However, the central claim is not supported by the paper's own aggregate results on MasakhaNER2, the benchmark that most directly represents low-resource African languages; as a result, the claimed significance is not established. The study remains a useful contribution to the engineering literature on annotation projection, with a clear formulation and broad evaluation, but the empirical conclusions need substantial qualification.

major comments (3)
  1. [Abstract and §5 Conclusion vs Table 1] The central claim that the proposed method 'surpasses existing projection-based methods in low-resource settings' and 'generally outperformed previous word-to-word alignment-based projection methods' is contradicted by Table 1 on MasakhaNER2. The best proposed variants, n-gram cand. SimAlign (66.4 macro-F1) and n-gram cand. AWESoME (66.7), are both below the heuristic AWESoME baseline (67.3), which Table 1 itself underlines as the best projection-based approach; the NER-candidate variants are substantially lower (63.0 and 62.5). Since MasakhaNER2 is the low-resource benchmark on which the 'low-resource' claim rests, the assertion of general superiority is unsupported by the macro-averaged results. The paper should either restrict the claim to the XTREME results, provide statistical testing or confidence intervals, or report a language-level analysis that justifies the conclusion despite the negative aggregate.
  2. [§4 Experiments] The heuristic baseline used for comparison is not the original published method: the authors state that they 'enhanced this heuristic by introducing a word count ratio threshold of 0.8'. No comparison against the unmodified heuristic algorithm is reported, and no ablation of the 0.8 threshold is provided. Because the conclusion is specifically about outperforming 'previous' heuristic projection methods, the baseline must be the original method as published; otherwise the 0.8 threshold could be responsible for part of the observed difference.
  3. [Abstract and Table 1] The abstract's claim that data-based transfer 'can outperform multilingual language models for low-resource languages' is only weakly supported and is dataset-dependent. On XTREME, model transfer has the highest average (50.9) among all rows, while the best proposed projection method (NER cand. SimAlign) reaches 46.4. On MasakhaNER2, projection methods do beat model transfer, but the opposite is true on XTREME. The manuscript should state this trade-off explicitly instead of presenting the claim as a general finding.
minor comments (5)
  1. [Title] The title contains an unwanted space in 'Cross-Ling ual'; it should read 'Cross-Lingual'.
  2. [§1 Introduction] The sentence 'achieves comparable and, in some cases, higher performance of multilingual language models' is ungrammatical and ambiguous; it should be rephrased as, for example, 'achieves performance comparable to, and in some cases higher than, multilingual language models'.
  3. [§4 Experiments and Table 1] Table 1 reports only selected languages and aggregate averages for XTREME and MasakhaNER2, while full per-language results are only available in the GitHub repository. Including the complete per-language table in the appendix would improve reproducibility and allow readers to verify the language-level claims made in the text.
  4. [Appendix] There is a typo in 'the proposed methods perfom worst' in the appendix; it should be 'perform worst'.
  5. [§3.2 Candidate Matching] The discussion of complexity is left as an open question, but the appendix notes that excluding the second constraint set reduces the problem to maximum weight independent set on interval graphs. A sentence in the main text clarifying that the greedy algorithm is used because the full problem's polynomial-time status is unknown would be helpful.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the proposed projection method is evaluated against external benchmarks using external models, and no fitted parameter is renamed as a prediction.

full rationale

The paper's contribution is a construction: a candidate-matching projection method with an alignment-based score (Eq. 2) and a greedy/ILP assignment. The score is computed from word alignments and span lengths, not from benchmark labels or from the quantities being predicted. No parameter is fitted to the evaluation data; the only manually set threshold (0.8) appears in the reimplemented heuristic baseline, not in the proposed method. The one self-citation (Torge et al., 2023) is related-work context and is not load-bearing for any central claim. The method is evaluated on external benchmarks (XTREME, MasakhaNER2, Europarl) with external models (NLLB-200, SimAlign, AWESoME, XLM-R-Large), so the empirical claims are independently testable. Whether the conclusion 'generally outperformed' is actually supported by Table 1 is a consistency/correctness concern, not a circularity concern. No load-bearing step reduces to its own inputs by construction.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

No free parameters are introduced in the proposed method itself; the only hand-tuned value is a threshold in a reimplemented baseline. The method's validity rests on the quality of the translation, NER, and alignment components, all of which are domain assumptions acknowledged in the Limitations section.

free parameters (1)
  • word_count_ratio_threshold = 0.8
    Manually chosen in the reimplemented heuristic baseline (Section 4) to merge misaligned single words. It is not part of the proposed method, but it alters the baseline strength and therefore the comparison.
assumptions (5)
  • domain assumption SimAlign/AWESoME word alignments between source and target sentences are accurate enough for matching scores to identify correct target spans.
    Used in Eq. 2 to define the matching score; the paper's Limitations section admits alignment quality varies across languages and bounds projection quality.
  • domain assumption The NLLB-200-3.3B translation model preserves named entities during translation and back-translation.
    The full pipeline requires entities to survive translation; Limitations state translation errors propagate through the pipeline.
  • domain assumption The English-fine-tuned XLM-R-Large NER model produces sufficiently correct source labels and target candidate spans for projection.
    The model is used as source NER and candidate extractor; Limitations note incorrect predictions or omissions cannot be fully corrected in projection.
  • domain assumption N-gram (all continuous word sequences) or NER-based candidate enumeration covers all true entity spans in the target sentence.
    The matching can only select from the candidate set T; if a true entity is absent from T, projection cannot recover it.
  • domain assumption The greedy approximation to the ILP in Eq. 1 yields solutions close enough to the optimum for the reported comparisons.
    The paper uses a greedy algorithm and acknowledges in Limitations that it may not achieve global optima.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Revisiting Projection-based Data Transfer for Cross-Lingual Named Entity Recognition in Low-Resource Languages." pith.science (2026). https://pith.science/paper/EURQQ6OL

@misc{pith2026250118750,
  author       = {Pith},
  title        = {Pith review of: Revisiting Projection-based Data Transfer for Cross-Lingual Named Entity Recognition in Low-Resource Languages},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EURQQ6OL}},
  note         = {Machine review of arXiv:2501.18750}
}
read the original abstract

Cross-lingual Named Entity Recognition (NER) leverages knowledge transfer between languages to identify and classify named entities, making it particularly useful for low-resource languages. We show that the data-based cross-lingual transfer method is an effective technique for crosslingual NER and can outperform multilingual language models for low-resource languages. This paper introduces two key enhancements to the annotation projection step in cross-lingual NER for low-resource languages. First, we explore refining word alignments using back-translation to improve accuracy. Second, we present a novel formalized projection approach of matching source entities with extracted target candidates. Through extensive experiments on two datasets spanning 57 languages, we demonstrated that our approach surpasses existing projectionbased methods in low-resource settings. These findings highlight the robustness of projection-based data transfer as an alternative to model-based methods for crosslingual named entity recognition in lowresource languages.

Figures

Figures reproduced from arXiv: 2501.18750 by the authors.

Figure 1
Figure 1. Proposed improvements to projection-based cross [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 15 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Alabi, Shamsuddeen H

    David Ifeoluwa Adelani, Graham Neubig, Sebastian Ruder, Shruti Rijhwani, Michael Beukman, Chester Palen-Michel, Constantine Lignos, Jesujoba O. Alabi, Shamsuddeen H. Muhammad, Peter Nabende, Cheikh M. Bamba Dione, Andiswa Bukula, Rooweither Mabuya, Bonaventure F. P. Dossou, Blessing Sibanda, Happy Buzaaba, Jonathan Mukiibi, Godson Kalipe, Derguene Mbaye, ...

  4. [4]

    Rodrigo Agerri, Yiling Chung, Itziar Aldabe, Nora Aranberri, Gorka Labaka, and German Rigau. 2018. https://aclanthology.org/L18-1557 Building named entity recognition taggers via parallel corpora . In Proceedings of the Eleventh International Conference on Language Resources and Evaluation ( LREC 2018) , Miyazaki, Japan. European Language Resources Associ...

  5. [5]

    Yang Chen, Chao Jiang, Alan Ritter, and Wei Xu. 2023. https://doi.org/10.18653/v1/2023.findings-acl.357 Frustratingly easy label projection for cross-lingual transfer . In Findings of the Association for Computational Linguistics: ACL 2023, pages 5775--5796, Toronto, Canada. Association for Computational Linguistics

  6. [6]

    Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzm \'a n, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. https://doi.org/10.18653/v1/2020.acl-main.747 Unsupervised cross-lingual representation learning at scale . In Proceedings of the 58th Annual Meeting of the Association for Comp...

  7. [7]

    Marta R Costa-juss \`a , James Cross, Onur C elebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672

  8. [8]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long a...

Show all 31 references
  1. [9]

    Zi-Yi Dou and Graham Neubig. 2021. https://doi.org/10.18653/v1/2021.eacl-main.181 Word alignment by fine-tuning embeddings on parallel corpora . In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2...

  2. [10]

    Chris Dyer, Victor Chahuneau, and Noah A. Smith. 2013. https://aclanthology.org/N13-1073 A simple, fast, and effective reparameterization of IBM model 2 . In Proceedings of the 2013 Conference of the North A merican Chapter of the Association for Computational Linguistics: Hum...

  3. [11]

    Eberhard, Gary F

    David M. Eberhard, Gary F. Simons, and Charles D. Fennig. 2020. http://www.ethnologue.com Ethnologue: Languages of the World , 23 edition. SIL International, Dallas

  4. [12]

    Hao Fei, Meishan Zhang, and Donghong Ji. 2020. https://doi.org/10.18653/v1/2020.acl-main.627 Cross-lingual semantic role labeling with high-quality translated training corpus . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 70...

  5. [13]

    Iker Garc \' a-Ferrero, Rodrigo Agerri, and German Rigau. 2022. https://doi.org/10.18653/v1/2022.findings-emnlp.478 Model and data transfer for cross-lingual sequence labelling in zero-resource settings . In Findings of the Association for Computational Linguistics: EMNLP 2022...

  6. [14]

    Iker Garc \' a-Ferrero, Rodrigo Agerri, and German Rigau. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.1015 T -projection: High quality annotation projection for sequence labeling tasks . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages ...

  7. [15]

    Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson. 2020. Xtreme: A massively multilingual multi-task benchmark for evaluating cross-lingual generalisation. In International Conference on Machine Learning, pages 4411--4421. PMLR

  8. [16]

    REBECCA HWA, PHILIP RESNIK, AMY WEINBERG, CLARA CABEZAS, and OKAN KOLAK. 2005. https://doi.org/10.1017/S1351324905003840 Bootstrapping parsers via syntactic projection across parallel texts . Natural Language Engineering, 11(3):311–325

  9. [17]

    Masoud Jalili Sabet, Philipp Dufter, Fran c ois Yvon, and Hinrich Sch \"u tze. 2020. https://doi.org/10.18653/v1/2020.findings-emnlp.147 S im A lign: High quality word alignments without parallel training data using static and contextualized embeddings . In Findings of the Ass...

  10. [18]

    Duong Minh Le, Yang Chen, Alan Ritter, and Wei Xu. 2024. https://api.semanticscholar.org/CorpusID:267412651 Constrained decoding for cross-lingual label projection . ArXiv, abs/2402.03131

  11. [19]

    Linlin Liu, Bosheng Ding, Lidong Bing, Shafiq Joty, Luo Si, and Chunyan Miao. 2021. https://doi.org/10.18653/v1/2021.acl-long.453 M ul DA : A multilingual data augmentation framework for low-resource cross-lingual NER . In Proceedings of the 59th Annual Meeting of the Associat...

  12. [20]

    J.F. Maho. 1999. https://books.google.de/books?id=34gLAQAAMAAJ A Comparative Study of Bantu Noun Classes . Acta Universitatis Gothoburgensis: Orientalia et Africana Gothoburgensia. Acta Universitatis Gothoburgensis

  13. [21]

    Kalyani Pakhale. 2023. http://arxiv.org/abs/2309.14084 Comprehensive overview of named entity recognition: Models, domain-specific applications and challenges

  14. [22]

    Madhumangal Pal and G. P. Bhattacharjee. 1996. https://doi.org/10.1080/00207169608804486 A sequential algorithm for finding a maximum weight K-independent set on interval graphs . Int. J. Comput. Math., 60(3-4):205--214

  15. [23]

    Tanmay Parekh, I-Hung Hsu, Kuan-Hao Huang, Kai-Wei Chang, and Nanyun Peng. 2024. https://doi.org/10.18653/v1/2024.naacl-long.321 Contextual label projection for cross-lingual structured prediction . In Proceedings of the 2024 Conference of the North American Chapter of the Ass...

  16. [24]

    Alberto Poncelas, Maksim Tkachenko, and Ohnmar Htun. 2023. https://doi.org/10.18653/v1/2023.semeval-1.239 Sakura at S em E val-2023 task 2: Data augmentation via translation . In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1718--...

  17. [25]

    Sebastian Ruder, Jonathan Clark, Alexander Gutkin, Mihir Kale, Min Ma, Massimo Nicosia, Shruti Rijhwani, Parker Riley, Jean-Michel Sarr, Xinyi Wang, John Wieting, Nitish Gupta, Anna Katanova, Christo Kirov, Dana Dickinson, Brian Roark, Bidisha Samanta, Connie Tao, David Adelan...

  18. [26]

    Henning Sch \"a fer, Ahmad Idrissi-Yaghir, Peter Horn, and Christoph Friedrich. 2022. https://doi.org/10.18653/v1/2022.clinicalnlp-1.6 Cross-language transfer of high-quality annotations: Combining neural machine translation with cross-linguistic span alignment to apply NER to...

  19. [27]

    J \"o rg Tiedemann. 2015. https://aclanthology.org/W15-1824 Improving the cross-lingual projection of syntactic dependencies . In Proceedings of the 20th Nordic Conference of Computational Linguistics ( NODALIDA 2015) , pages 191--199, Vilnius, Lithuania. Link \"o ping Univers...

  20. [28]

    Tjong Kim Sang and Fien De Meulder

    Erik F. Tjong Kim Sang and Fien De Meulder. 2003. https://aclanthology.org/W03-0419 Introduction to the C o NLL -2003 shared task: Language-independent named entity recognition . In Proceedings of the Seventh Conference on Natural Language Learning at HLT - NAACL 2003 , pages 142--147

  21. [29]

    Sunna Torge, Andrei Politov, Christoph Lehmann, Bochra Saffar, and Ziyan Tao. 2023. https://doi.org/10.18653/v1/2023.bsnlp-1.1 Named entity recognition for low-resource languages - profiting from language families . In Proceedings of the 9th Workshop on Slavic Natural Language...

  22. [30]

    Jannis Vamvas and Rico Sennrich. 2022. https://doi.org/10.18653/v1/2022.findings-emnlp.15 NMTS core: A multilingual analysis of translation-based text similarity measures . In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 198--213, Abu Dhabi, Uni...

  23. [31]

    Jian Yang, Shaohan Huang, Shuming Ma, Yuwei Yin, Li Dong, Dongdong Zhang, Hongcheng Guo, Zhoujun Li, and Furu Wei. 2022. https://doi.org/10.18653/v1/2022.findings-emnlp.34 CROP : Zero-shot cross-lingual named entity recognition with multilingual labeled sequence translation . ...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.