Pith. sign in

REVIEW 1 major objections 5 minor 22 references

Cross-Lingual Transfer for Low-Resource Natural Language Processing

T0 review · 1 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This thesis establishes a generate-then-rank annotation projection method and a constrained decoding algorithm that together improve cross-lingual sequence labeling for low-resource languages.

desk verdict Real methods and useful open resources, but the intrinsic evaluation is compromised by training/evaluation overlap and the headline margin needs to be recomputed. read the letter →

arxiv 2502.02722 v1 pith:WZOUJWAZ submitted 2025-02-04 cs.CL

Iker García-Ferrero This is my paper · ORCID
classification cs.CL
keywords cross-lingualtransferannotationprojectionsequencelabelinglow-resourcelanguagesconstraineddecodingtext-to-textlanguagemodelsnamedentityrecognitionmultilingualmedicalNLP
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This thesis tries to establish two mechanisms for cross-lingual transfer in low-resource sequence labeling. T-Projection is an annotation projection method that generates candidate target-language spans with a fine-tuned multilingual text-to-text model and ranks them by machine-translation probability; it outperforms previous projection methods by 8.6 average $F_1$ points across Named Entity Recognition, Opinion Target Extraction, and Argument Mining. The second mechanism is a constrained decoding algorithm that forces text-to-text language models to copy the input sentence and emit well-formed labels, enabling mT0-XL to beat encoder-only models in zero-shot cross-lingual sequence labeling. The same methods are applied to build the first open multilingual text-to-text medical model, showing that the approach reaches real-world low-resource domains.

What carries the argument

The load-bearing machinery is T-Projection's two-step pipeline: candidate generation, in which a fine-tuned multilingual text-to-text model (mT5) sees the unlabeled target sentence with category tags taken from the source sentence and generates many candidate spans; and candidate selection, in which a machine-translation model scores each candidate by normalized, symmetrized translation probability and chooses the best non-overlapping assignment. The second mechanism is the constrained-decoding finite-state automaton, which restricts each decoding step to tokens that copy the next input word or open or close a label tag, implemented as constrained beam search so the output is always the input sentence with valid tags. The first mechanism removes reliance on word alignments; the second removes reliance on a model's free-form output discipline.

What would settle it

Run T-Projection on a language pair or domain not covered by the thesis, with human-projected gold labels, and measure how often the correct target span appears in the generator's top-$k$ candidate lists. If that recall is well below the 98.0 average $F_1$ reported in the thesis, the selection step will not be able to compensate, and the claimed margin over SimAlign should shrink proportionally.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that cross-lingual sequence labeling improves when the two standard tools are replaced: word-alignment-based annotation projection gives way to a generate-then-rank method, and free-form generation by text-to-text models gives way to constrained decoding. T-Projection fine-tunes a multilingual text-to-text model to propose candidate target-language spans for each labeled category in the parallel sentence, then scores those candidates with machine-translation probabilities from M2M100 or NLLB200, symmetrized across translation directions, and selects the best non-overlapping spans. In intrinsic evaluation against manually projected gold labels across OTE, NER, and Argument Mining in five languages, T-Projection averages 93.9 $F_1$ and outperforms the second-best system, SimAlign, by 8.6 points; in the extrinsic African-languages evaluation it beats zero-shot encoder transfer by 15.6 average $F_1$ points. The constrained-decoding contribution defines a finite-state automaton that forces a text-to-text model to copy the input sentence word-by-word and insert or close only valid HTML-style tags; applied to mT0-XL, this eliminates hallucinated words and unparseable outputs and outperforms both unconstrained beam search and encoder-only baselines in zero-shot cross-lingual sequence labeling, most strongly for languages morphologically distant from English. The medical case study extends the same data- and model-transfer methods to build the first open multilingual text-to-text model for the medical domain.

Load-bearing premise

T-Projection's gains depend on the fine-tuned multilingual generator placing the correct translation of each labeled source span among its top candidates for the parallel sentence; if candidate recall fails, the machine-translation ranking step cannot recover, and the paper's own upper-bound experiment puts that recall ceiling at 98.0 average $F_1$.

Editorial extensions

If this is right

  • Data-based transfer becomes the recommended route when no large multilingual encoder has strong proficiency in the target language: T-Projection's projected data beat zero-shot encoder transfer in six of eight African languages, by 15.6 average $F_1$ points.
  • Annotation projection for long, structured spans no longer needs word alignments; on Argument Mining projection T-Projection reaches 96.0 $F_1$, 9.4 points above the next-best system.
  • Text-to-text models become viable for zero-shot cross-lingual sequence labeling once decoding is constrained to copy the input words and produce well-formed tags; mT0-XL with constrained beam search beats unconstrained decoding and encoder-only baselines, especially for languages distant from English.
  • The same transfer toolkit carries into new domains: a multilingual text-to-text medical model, trained on compiled medical corpora and evaluated on sequence labeling and question answering, shows that low-resource medical NLP can use the same data- and model-transfer routes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own upper-bound result implies that future gains should come from improving candidate recall: at 98.0 average $F_1$ the selection step is already near-ceiling, so the bottleneck is the generator's ability to put the right span in its top candidates.
  • A testable extension is to carry the generate-then-rank scheme to other span-level tasks, such as event extraction or coreference, where word alignments are known to be fragile.
  • Constrained decoding might be pushed further by training the generator to expect the constraint, rather than imposing it only at inference, which could reduce the language-mixing errors that still appear in zero-shot outputs.
  • Because T-Projection still needs labeled data in a source language and a good translation system into the target, its practical reach is bounded by those resources; the thesis's African-languages results suggest the boundary is wider than word-alignment methods allowed, but not infinite.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 5 minor

Summary. This PhD thesis addresses cross-lingual transfer learning for sequence labeling tasks (NER, OTE, Argument Mining) in low-resource languages. It makes three main contributions: (1) a comparative study of data-based versus model-based transfer with an error analysis (Chapter 3); (2) T-Projection, a two-stage annotation projection method that generates candidates with a fine-tuned mT5 model and selects among them using MT-based translation probabilities (Chapter 4); and (3) a constrained decoding algorithm for text-to-text LLMs that enforces the output structure of sequence labeling, evaluated on NER, OTE, and Event Extraction across up to 25 languages (Chapter 5). Chapter 6 applies these ideas to the medical domain by releasing Multilingual Medical Corpus and Medical mT5. The headline claims are that T-Projection outperforms prior projection methods by 8.6 F1 points on average in intrinsic evaluation and that constrained decoding lets mT0-XL beat encoder-only models in zero-shot cross-lingual sequence labeling.

Significance. If the results hold, this is a practically significant body of work for low-resource NLP. The thesis releases open-source software, datasets, and models (T-Projection, Medical mT5, Easy Label Projection), which is a genuine strength for reproducibility and community adoption. The two-step projection design, with separate analysis of candidate generation and selection, is a well-structured contribution that improves over alignment-based methods, especially for long span projections (Argument Mining). The constrained decoding algorithm addresses a real failure mode of text-to-text models (hallucinations and invalid output structure) and shows consistent gains over unconstrained decoding. The comparison between data and model transfer in Chapter 3 and the application to African languages in Chapters 4 and 5 are useful empirical anchors for practitioners.

major comments (1)
  1. [Section 5.3.2, final paragraph] The statement that 'the constrained beam search will always yield an output that is at least as good as, if not superior to, unconstrained beam search' is logically incorrect. Constraining the search space to valid sequences can exclude the highest-probability decoding, and an unconstrained output that is invalid may still contain correct entity spans that yield a higher F1 after parsing than any valid constrained output. What is guaranteed is that constrained decoding produces a well-formed output and removes hallucinated tokens outside the input. The empirical results in Tables 5.4-5.8 are not in question, but this theoretical claim should be removed or revised accordingly.
minor comments (5)
  1. [Tables 4.2, 4.3, 4.5, 5.4-5.8] These central tables report single-run F1 scores without variance estimates, whereas Chapter 3 reports five-seed averages with standard deviations. This makes it difficult to assess small per-language differences, e.g., Igbo 71.4 vs 70.9 in Table 4.5.
  2. [Section 1.4.1] Chapter 5 (constrained decoding) is not listed in the scientific contributions included in the thesis; the provenance of this work should be clarified, whether it is an unpublished chapter or it has appeared elsewhere.
  3. [Section 4.6.2] The claim that 'these are the best zero-shot results achieved for MasakhaNER2.0' should be qualified, since the evaluation covers 8 of the 20 languages and a specific training setup (English CoNLL plus NLLB translations); as written it could be misread as a claim over the full benchmark.
  4. [Section 4.5.2] The 'upper bound' defined by selecting the correct candidate when present and otherwise the most probable candidate mixes candidate recall and selection accuracy; please define it explicitly as a practical ceiling rather than a pure recall bound.
  5. [Throughout] Several typos and small errors: Chapter 6 title 'Domain-Spacific' should be 'Domain-Specific'; Figure 5.5 caption says 'unconstrained and unconstrained beam search' instead of 'constrained and unconstrained'; Section 2.2 has 'Comercial' for 'Commercial'; Figure 2.15 caption has 'coss-lingual'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central results are empirical comparisons against external and manually constructed benchmarks; the noted train/evaluation overlap is a data-leakage concern, not a definitional reduction.

full rationale

T-Projection's headline results (8.6 F1 points over SimAlign in intrinsic evaluation; 3.6 points in the MasakhaNER2.0 extrinsic evaluation) are empirical measurements against human-annotated or human-projected gold standards, not quantities derived by construction from the method's own inputs. Candidate generation uses mT5 fine-tuned on source-language labels, and candidate selection uses M2M100/NLLB translation probabilities; these are independent components whose combination is evaluated, not assumed. The oracle upper bound of 98.0 F1 (Section 4.5.2) is explicitly defined as always selecting the correct candidate when present, so it is a measured ceiling on candidate recall rather than a hidden fitted parameter. The constrained-decoding chapter and Medical mT5 chapter are likewise validated on held-out external test sets (MasakhaNER2.0, ABSA test splits, BioASQ). The one overlap worth recording is in the intrinsic evaluation of Section 4.5: T-Projection is trained on the full English ABSA 2016, CoNLL03+Europarl, and AbstRCT data, while the manual projection gold standards were built from translations of those same English sources. This may inflate intrinsic scores and is a legitimate correctness/leakage question, but it is not circularity: the gold labels come from human annotators projecting labels onto target-language translations, and T-Projection's outputs are not algebraically or definitionally equal to those gold labels. Self-citations and reuse of the authors' Chapter 3 manual projections as an evaluation resource are present, but no load-bearing argument reduces to a self-cited theorem, ansatz, or fitted parameter. Therefore, no circular step is established.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

Central claims rest on strong external benchmarks, but the method introduces no new physical or formal entities. The main free choices are the number of candidates and model scale. Several domain assumptions about translation preserving annotations and about mT5 zero-shot recall are load-bearing.

free parameters (2)
  • number_of_projection_candidates = 100
    Section 4.2.1 says 100 beams was decided after preliminary analysis; Section 4.5.3 states 10 to 25 is optimal, so the reported main results use a tuned, suboptimal setting.
  • mT5_model_size = 3B (mT5-xl)
    T-Projection uses the 3B mT5; Table 4.4 shows 738M is competitive for OTE and NER but not AM, so model scale is a chosen capacity parameter.
assumptions (4)
  • domain assumption Annotations are preserved by translation, including span boundaries and labels.
    Stated in Section 2.2.1 as the assumption of data transfer; all projection methods rely on it.
  • domain assumption mT5, fine-tuned on source-language labeled data, can generate the correct target-language span among its top candidates for unseen language pairs.
    Core of T-Projection candidate generation in Section 4.2.1; if candidate recall fails, selection cannot recover. The upper bound in Section 4.5.2 (98.0 avg F1) shows this limits performance.
  • domain assumption Machine translation systems M2M100 and NLLB200 provide reliable translation-equivalence scores for candidate selection.
    T-Projection candidate selection uses NMTScore probabilities from these MT models, described in Section 4.3.
  • domain assumption Fine-tuning on English data transfers to target languages for zero-shot model transfer.
    Standard assumption for model-based transfer, used throughout Chapters 3 and 5.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Cross-Lingual Transfer for Low-Resource Natural Language Processing." pith.science (2026). https://pith.science/paper/WZOUJWAZ

@misc{pith2026250202722,
  author       = {Pith},
  title        = {Pith review of: Cross-Lingual Transfer for Low-Resource Natural Language Processing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WZOUJWAZ}},
  note         = {Machine review of arXiv:2502.02722}
}
read the original abstract

Natural Language Processing (NLP) has seen remarkable advances in recent years, particularly with the emergence of Large Language Models that have achieved unprecedented performance across many tasks. However, these developments have mainly benefited a small number of high-resource languages such as English. The majority of languages still face significant challenges due to the scarcity of training data and computational resources. To address this issue, this thesis focuses on cross-lingual transfer learning, a research area aimed at leveraging data and models from high-resource languages to improve NLP performance for low-resource languages. Specifically, we focus on Sequence Labeling tasks such as Named Entity Recognition, Opinion Target Extraction, and Argument Mining. The research is structured around three main objectives: (1) advancing data-based cross-lingual transfer learning methods through improved translation and annotation projection techniques, (2) developing enhanced model-based transfer learning approaches utilizing state-of-the-art multilingual models, and (3) applying these methods to real-world problems while creating open-source resources that facilitate future research in low-resource NLP. More specifically, this thesis presents a new method to improve data-based transfer with T-Projection, a state-of-the-art annotation projection method that leverages text-to-text multilingual models and machine translation systems. T-Projection significantly outperforms previous annotation projection methods by a wide margin. For model-based transfer, we introduce a constrained decoding algorithm that enhances cross-lingual Sequence Labeling in zero-shot settings using text-to-text models. Finally, we develop Medical mT5, the first multilingual text-to-text medical model, demonstrating the practical impact of our research on real-world applications.

Figures

Figures reproduced from arXiv: 2502.02722 by the authors.

Figure 1
Figure 1. [PITH_FULL_IMAGE:figures/full_fig_p026_1.png] view at source ↗
Figure 1
Figure 1. [PITH_FULL_IMAGE:figures/full_fig_p028_1.png] view at source ↗
Figure 2
Figure 2. [PITH_FULL_IMAGE:figures/full_fig_p042_2.png] view at source ↗
Figures from the paper (50 more)
Figure 2
Figure 2. Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p043_2.png]
Figure 2
Figure 2. Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p044_2.png]
Figure 2
Figure 2. Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p046_2.png]
Figure 2
Figure 2. Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p047_2.png]
Figure 2
Figure 2. Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p048_2.png]
Figure 2
Figure 2. Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p049_2.png]
Figure 2
Figure 2. Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p050_2.png]
Figure 2
Figure 2. Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p051_2.png]
Figure 2
Figure 2. Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p052_2.png]
Figure 2.10
Figure 2.10. Figure 2.10: Traditionally, most research in this area has focused on automatically [PITH_FULL_IMAGE:figures/full_fig_p052_2_10.png]
Figure 2
Figure 2. Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p053_2.png]
Figure 2
Figure 2. Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p054_2.png]
Figure 2
Figure 2. Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p055_2.png]
Figure 2
Figure 2. Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p057_2.png]
Figure 3
Figure 3. Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p062_3.png]
Figure 3
Figure 3. Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p063_3.png]
Figure 3
Figure 3. Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p064_3.png]
Figure 3
Figure 3. Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p065_3.png]
Figure 3
Figure 3. Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p066_3.png]
Figure 3
Figure 3. Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p069_3.png]
Figure 3
Figure 3. Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p074_3.png]
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p085_4.png]
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p086_4.png]
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p088_4.png]
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p090_4.png]
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p091_4.png]
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p092_4.png]
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p097_4.png]
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p098_4.png]
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p107_5.png]
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p110_5.png]
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p111_5.png]
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p113_5.png]
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p121_5.png]
Figure 5
Figure 5. Figure 5: indicates that word splitting and hallucinations correlate with the [PITH_FULL_IMAGE:figures/full_fig_p123_5.png]
Figure 5
Figure 5. Figure 5: , show that as the mT0 model’s parameter count increases, the F1 score [PITH_FULL_IMAGE:figures/full_fig_p124_5.png]
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p125_5.png]
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p134_6.png]
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p136_6.png]
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p137_6.png]
Figure 1
Figure 1. Figure 1: In the data-based transfer approach we trans [PITH_FULL_IMAGE:figures/full_fig_p189_1.png]
Figure 2
Figure 2. Figure 2: Illustration of the translation and annotation [PITH_FULL_IMAGE:figures/full_fig_p192_2.png]
Figure 3
Figure 3. Figure 3: Sequence Labelling tasks used in our experi [PITH_FULL_IMAGE:figures/full_fig_p192_3.png]
Figure 4
Figure 4. Figure 4: Application used to manually annotate the projections [PITH_FULL_IMAGE:figures/full_fig_p202_4.png]
Figure 1
Figure 1. Figure 1: T-Projection two-step method to project se [PITH_FULL_IMAGE:figures/full_fig_p203_1.png]
Figure 2
Figure 2. Figure 2: Illustration of the candidate generation step. [PITH_FULL_IMAGE:figures/full_fig_p205_2.png]
Figure 3
Figure 3. Figure 3: Candidate selection: candidates are scored [PITH_FULL_IMAGE:figures/full_fig_p206_3.png]
Figure 4
Figure 4. Figure 4: Sequence labeling tasks in our experiments [PITH_FULL_IMAGE:figures/full_fig_p207_4.png]
Figure 5
Figure 5. Figure 5: F1 score when generating a different number [PITH_FULL_IMAGE:figures/full_fig_p215_5.png]
Figure 1
Figure 1. Figure 1: Text-to-Text representation of the Se￾quence Labeling task. Given an input sentence, the model is expected to generate the same sen￾tence annotated with html-style tags. the models. 5.2. Question Answering We use the BioASQ-6B English Question Answer￾ing dataset (Tsats…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 17 canonical work pages

  1. [1]

    With this aim in mind, in the last few years a number of large language models (LLMs) have been trained or adapted to the medi- cal domain

    Introduction As it is the case for many application domains, there is an increasing interest in applying Artificial In- telligence (AI) and Natural Language Processing (NLP) techniques to assist medical experts in their everyday activities. With this aim in mind, in the last few years a number of large language models (LLMs) have been trained or adapted t...

    work page 2019

  2. [2]

    , 2022; Wu et al

    Related Work As it has been the case in most application do- mains, Large Language Models (LLMs) have facili- tated huge improvements in the state-of-the-art for medical NLP tasks ( Singhal et al. , 2022; Wu et al. , 2023; Mayer et al. , 2021). The most popular ap- proaches are those that use models pre-trained on medical corpora such as SciBERT ( Beltagy...

    work page 2022

  3. [3]

    This is even more challenging for non-English lan- guages, as the availability of data for other lan- guages is in general more restricted

    Compiling a Multilingual Corpus for the Medical Domain Obtaining good quality medical corpora is usually difficult due to the sensitive nature of the data. This is even more challenging for non-English lan- guages, as the availability of data for other lan- guages is in general more restricted. Despite these issues, we have successfully gathered and curat...

    work page 2022

  4. [5]

    , 8Sourced from https://app.electricitymaps

    Generating New Multilingual Benchmarks The lack of multilingual evaluation benchmarks for the medical domain motivated us to generate new evaluation data for our languages of interest, as only the relatively small E3C ( Magnini et al. , 8Sourced from https://app.electricitymaps. com/map Medical-mT5-large Medical-mT5-xl Param. no. 738M 3B Sequence Lenght 1...

    work page

  5. [6]

    and prism (Thompson and Post, 2020) as model for computing translation probabilities. We also experiment with using LASER 2.0 (Artetxe and Schwenk, 2019) sentence representations in- 15215 OTE NER AM Average System Data Augmentation Backbone ES FR RU ES DE IT ES Giza++ (Och and Ney, 2003) 0 mBERT 76.2 73.8 78.2 71.4 66.6 65.7 86.4 76.8 FastAlign (Dyer et ...

    work page 2023

  6. [7]

    <Disease> Patient with dilated car- diomyopathy

    Experimental Results In this section, we report on the performance of Medical mT5 and of the baselines in the sequence labelling tasks across different settings. Due to space constraints, we only report the best perform- ing results. Single Task Monolingual Supervised Results: The results when fine-tuning and evaluating the models for each dataset and lan...

    work page 2022

  7. [8]

    Its development has required the compilation of a new 3B word corpus in English, French, Italian and Spanish specific to the medi- cal domain

    Conclusion In this paper we have presented Medical mT5, the first open source multilingual text-to-text LLM for the medical domain. Its development has required the compilation of a new 3B word corpus in English, French, Italian and Spanish specific to the medi- cal domain. Furthermore, motivated by the lack of multilingual benchmarks, we have generated e...

    work page 2022

  8. [9]

    These models, to name but a few, include SciFive ( Phan et al

    has resulted in a huge interest in adapting such LLMs to the medical domain. These models, to name but a few, include SciFive ( Phan et al. , 2021), and English T5 encoder-decoder model adapted to the scientific domain, and decoder mod- els such as BioGPT ( Luo et al. , 2022), Med-PaLM (Singhal et al., 2022), PMC-LLaMA (Wu et al., 2023) and ClinicalGPT ( ...

    work page 2021

Show all 22 references
  1. [10]

    Ethical Statement Our research in developing Medical mT5, a multilin- gual text-to-text model for the medical domain, has ethical implications that we acknowledge. Firstly, the broader impact of this work lies in its poten- tial to improve medical communication and under- stan...

    2022 arXiv

  2. [12]

    EDP French corpus, text format

    provides bilingual content from journals that address domains such as dentistry and life sciences. From this source, we downloaded the dataset labeled “EDP French corpus, text format”. Finally, Google Patents is a comprehensive repository of patent data from around the world. ...

    2022

  3. [15]

    This means that, given a text input, it learns to generate a text as output

    Experimental Setup Medical mT5 is a text-to-text model. This means that, given a text input, it learns to generate a text as output. Therefore, every evaluation task must be converted into a text-to-text format ( Xue et al. , 2021). In our experiments the output text is always...

    2021

  4. [18]

    Iker García-Ferrero is supported by a doctoral grant from the Basque Government (PRE_2021_2_0219)

    Acknowledgements HiTZ Center : This work has been supported by the following MCIN/AEI/10.13039/501100011033 projects: (i) Antidote (PCI2020-120717-2) and EU NextGenerationEU/PRTR (ii) DeepKnowledge (PID2021-127777OB-C21) and by FEDER, EU; (iii) DeepR3 (TED2021-130295B-C31) and...

    doi

  5. [21]

    In Proceed- ings of the 61st Annual Meeting of the Associ- ation for Computational Linguistics (Volume 1: Long Papers), ACL 2023, T oronto, Canada, July 9-14, 2023, pages 1082–1117

    Glot500: Scaling multilingual corpora and language models to 500 languages . In Proceed- ings of the 61st Annual Meeting of the Associ- ation for Computational Linguistics (Volume 1: Long Papers), ACL 2023, T oronto, Canada, July 9-14, 2023, pages 1082–1117. Association for Co...

    2023

  6. [22]

    Transformer-based argument mining for healthcare applications . In ECAI 2020 - 24th European Conference on Artificial Intelligence, 29 August-8 September 2020, Santiago de Com- postela, Spain, August 29 - September 8, 2020 - Including 10th Conference on Prestigious Ap- plicati...

    2020 arXiv

  7. [2001]

    In Proceed- ings of the First International Conference on Human Language Technology Research

    Inducing multilingual text analysis tools via robust projection across aligned corpora. In Proceed- ings of the First International Conference on Human Language Technology Research. Anar Yeginbergenova and Rodrigo Agerri. 2023. Cross- lingual argument mining in the medical dom...

    2023 arXiv

  8. [2011]

    In Pro- ceedings of the International Conference Recent Ad- vances in Natural Language Processing 2011, pages 118–124, Hissar, Bulgaria

    Building a multilingual named entity- annotated corpus using annotation projection. In Pro- ceedings of the International Conference Recent Ad- vances in Natural Language Processing 2011, pages 118–124, Hissar, Bulgaria. Association for Computa- tional Linguistics. Miquel Espl...

    2011 arXiv

  9. [2013]

    In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 644–648, Atlanta, Georgia

    A simple, fast, and effective reparameteriza- tion of IBM model 2. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 644–648, Atlanta, Georgia. Association for Computational ...

    2013

  10. [2017]

    Médecine

    and WikiMed, a corpus built ad-hoc from Wikipedia entries. T able 2 provides the details of the collected data, which amounts to ≈1B words. Source Words EMEA 13.6M PubMed 8.4M Medical Crawler 918M SPACC 350K UFAL 10.5M WikiMed 5.2M T able 2: Spanish data sources and word count...

  11. [2020]

    XTREME: A massively multilingual multi- task benchmark for evaluating cross-lingual gener- alisation. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research , pa...

    2020 arXiv

  12. [2021]

    was already available for all 4 languages. We focused on two different types of tasks: (i) a sequence labelling task, Argument Mining, con- sisting in detecting and classifying the argument component spans and their relations, (ii) Abstrac- tive Question Answering , where the ...

    2021

  13. [2022]

    In Find- ings of the Association for Computational Linguis- tics: EMNLP 2022, pages 6403—-6416, Abu Dhabi, United Arab Emirates

    Model and data transfer for cross-lingual se- quence labelling in zero-resource settings. In Find- ings of the Association for Computational Linguis- tics: EMNLP 2022, pages 6403—-6416, Abu Dhabi, United Arab Emirates. Association for Computa- tional Linguistics. Pengcheng He,...

    2022

  14. [2023]

    In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023

    DeBERT aV3: Improving DeBERT a us- ing ELECTRA-style pre-training with gradient- disentangled embedding sharing. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan A...

    2023 arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.