Pith. sign in

REVIEW 4 major objections 7 minor 42 references

Fine-Tuning LLMs for Low-Resource Dialect Translation: The Case of Lebanese

T0 review · 4 major / 7 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fine-tuning Aya23-8B on 3,000 authentic Lebanese sentences beats training on 140,000 translated ones.

desk verdict A useful new benchmark and a clean empirical setup are undermined by a headline claim that the paper's own Table 1 contradicts; the LW advantage only shows up on a small author-built test set of unclear independence. read the letter →

arxiv 2505.00114 v1 pith:XU4523D2 submitted 2025-04-30 cs.CL cs.AI

Silvana Yakhni , Ali Chehab This is my paper · ORCID
classification cs.CLcs.AI
keywords Lebanesedialectmachinetranslationlow-resourcelanguagescontrastivefine-tuningpromptingculturalauthenticitydataqualityoverquantityLebEvalbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that for translating the low-resource Lebanese dialect, a few thousand culturally authentic sentence pairs beat a much larger corpus of translated text, and that teaching a model to prefer good translations over bad ones improves quality further. It reports systematic fine-tuning experiments on the open-source Aya23-8B model, comparing basic translation instructions, contrastive instructions with chosen and rejected outputs, and grammar-hint instructions, trained on either the 3K-sentence Language Wave dataset or a 140K-sentence non-native collection. It also introduces LebEval, a 70-sentence evaluation set drawn from native Lebanese podcast content, and finds that models score much lower there than on the translated FLoRes benchmark. The best result comes from contrastive fine-tuning paired with contrastive prompting, which the paper reads as evidence against the assumption that more training data always means better translation.

What carries the argument

The load-bearing mechanism is contrastive instruction tuning. Each training example concatenates a chosen translation and a rejected translation, separated by '<rather than>' and accompanied by a hint field stating the preference, so the model learns to distinguish good from bad output; at inference the same idea is applied by including good and bad examples in a contrastive prompt. The other components are the Aya23-8B base model fine-tuned with efficient low-rank adaptation, the small culturally aware Language Wave dataset versus the larger non-native collection, and the LebEval benchmark used for evaluation.

What would settle it

Build a held-out Lebanese-English test set from a different source genre, such as social-media comments or transcribed news interviews, that does not share the podcast domain of Language Wave, and compare the LW-trained adapters against the NN-trained adapters on it; if the 140K-data models match or beat the 3K-data model there, the authenticity-over-quantity claim collapses into a domain-match effect. As a second check, have native speakers rate the best model's translations for idiom preservation and cultural appropriateness, and see whether the xCOMET gains track human preferences.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that data provenance matters more than data volume for dialectal translation: fine-tuning on the culturally aware Language Wave dataset, roughly 3,000 sentences from a Lebanese podcast, consistently outperforms fine-tuning on roughly 140,000 sentences of translated subtitles and travel feedback, across every prompting strategy tested. Among the training styles, contrastive instruction tuning, where the model sees a preferred translation and a rejected one in the same training example, outperforms plain translation tuning, and the gap widens when contrastive prompting is used at inference, with the Instruct-Cont-LW adapter reaching the highest xCOMET score of 74.4 on LebEval. The paper also claims that native-content evaluation is essential: the base model and all adapters score far higher on FLoRes than on LebEval, while curriculum learning without separate model stages yields no significant gains, and preference optimization with model-generated rejected translations underperforms contrastive fine-tuning.

Load-bearing premise

The claim that smaller culturally authentic data beats larger non-native data rests on LebEval being an independent test set, but the paper never shows that the LebEval sentences are disjoint from the Language Wave training sentences or that the two podcast-derived sources do not share a domain.

Editorial extensions

If this is right

  • A practical recipe for low-resource dialect translation emerges: collect a few thousand authentic parallel sentences from native speakers and prefer that over scraping large translated corpora.
  • Contrastive training data can be built cheaply by pairing golden translations with model-generated suboptimal outputs, and the same good/bad contrast can be moved into the prompt at test time.
  • Scores on translated benchmarks such as FLoRes overstate how well a model handles real dialectal text, so native-content benchmarks are a more demanding and more honest yardstick.
  • Curriculum learning and grammar-hint tuning did not pay off under the tested settings, so gains from those strategies should not be assumed without dedicated study.
  • Preference-alignment methods that rely on model-generated rejected examples can underperform simple contrastive fine-tuning in this setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because LebEval and Language Wave are both drawn from Lebanese podcast content, the small-data advantage may partly reflect domain match rather than cultural authenticity; a test set from a different genre would separate the two explanations.
  • The contrastive-tuning result suggests a broader principle for dialectal and cultural translation: explicit negative examples teach the model which fluent-sounding renderings are wrong, which plain parallel data cannot do.
  • The same design could be tested on other dialects or on culturally rich language pairs outside Arabic; if the pattern holds, it would generalize the authenticity-over-volume claim beyond Lebanese.
  • A human evaluation focused on idiom preservation and cultural appropriateness would tell whether the xCOMET gains correspond to the cultural nuance that matters for this translation task.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. This paper studies fine-tuning Aya23-8B for Lebanese-to-English translation, comparing adapters trained on a small native podcast-derived dataset (LW, about 3K sentences) with adapters trained on a larger translated corpus (NN, 140K sentences) under basic, contrastive, and grammar-hint instruction formats, with and without curriculum training, and under zero-shot, few-shot, and contrastive prompting. The paper also introduces LebEval, a 70-sentence evaluation set constructed by the authors from a Lebanese podcast, and evaluates all models with reference-free xCOMET on LebEval and a 500-sentence FLoRes subset. The headline claims are that LW-trained models consistently outperform NN-trained models, that contrastive fine-tuning paired with contrastive prompting is best, and that authentic evaluation benchmarks are necessary because FLoRes scores overestimate dialect translation quality.

Significance. Should these claims hold, the paper would contribute to low-resource dialect MT and to the 'data quality over quantity' debate. The experimental design is broad (three instruction types, two data sources, three prompting strategies, two curricula, and a preference-alignment comparison), the work targets an under-resourced dialect, and the authors state that code and data are released, which supports reproducibility. The introduction of LebEval is also a potentially useful resource. However, the central quality-over-quantity claim is contradicted by the paper's own FLoRes results, and the only benchmark that shows an aggregate LW advantage is LebEval, whose independence from the LW training distribution is not established. Section 7 candidly lists limitations for the synthetic grammar data and preference alignment, but it does not address either of these load-bearing issues. As presented, the empirical basis for the headline conclusion is not sufficient.

major comments (4)
  1. [Section 4.2, Table 1] The abstract and Section 4.2 claim that LW-trained models 'consistently' outperform NN-trained models, but the FLoRes columns in Table 1 contradict this. Across the 12 paired NN/LW comparisons (Instruct-MT, Instruct-Cont, CONT+MT, and Gram+CONT+MT in each prompting setting), NN-trained models score higher than LW-trained models in 11 cases (e.g., 0-shot Instruct-MT: 87.6 vs 86.9; C3-shot Instruct-Cont: 89.1 vs 87.4); LW leads only in the C3-shot Instruct-MT setting, by 0.3 points (87.0 vs 86.7). Even on LebEval the pattern is not universal: LW wins eight of the twelve paired comparisons, NN wins two (0-shot Instruct-Cont 71.8 vs 71.7; C3-shot Instruct-MT 71.1 vs 71.0), and two are ties. Because the paper's central claim is stated as a consistent advantage, the authors' own Table 1 is evidence against it, and Section 7 does not acknowledge the contradiction.
  2. [Sections 3.1 and 3.4] LebEval is not shown to be independent of the LW training data. Section 3.1 describes LW as approximately 3K sentences from 95 episodes of a Lebanese podcast; Section 3.4 describes LebEval as 70 sentences from 15 episodes of the 'Levantine Arabic Made Easier' podcast. The paper does not identify the LW podcast, does not provide episode IDs, does not perform or report sentence-level deduplication against LW, and contains no statement that the two corpora are disjoint. Because both corpora are native conversational Lebanese and the LW advantage is largest on LebEval, the observed result may be a train/test leakage or domain-match artifact rather than evidence for data quality over quantity. This missing overlap analysis is load-bearing and must be supplied before any conclusion can be drawn from LebEval.
  3. [Section 4.2, Table 1 (Vanilla row)] The paper interprets the raw xCOMET gap between FLoRes (85.5 for the base model) and LebEval (68.7) as evidence that culturally generic benchmarks misrepresent dialect translation quality. However, xCOMET scores are not calibrated across test sets; the two benchmarks differ in source genre, sentence length, transcription quality, and difficulty. A matched calibration or pairwise human judgments would be needed to support the claim that FLoRes is too easy or culturally generic. As written, the cross-benchmark conclusion in Section 4.2 is not supported.
  4. [Sections 3.4 and 4.2] No confidence intervals, bootstrap errors, or significance tests accompany any of the headline comparisons. LebEval contains 70 sentences, and many of the reported gaps are 0.1-0.7 xCOMET points (e.g., 0-shot Instruct-Cont on LebEval: 71.8 vs 71.7; C3-shot Instruct-MT on FLoRes: 86.7 vs 87.0). Without seed variance or interval estimates, the ranking of adapters and prompting strategies in Tables 1 and 2 cannot be distinguished from metric noise, especially because many configurations are compared at once.
minor comments (7)
  1. [Appendix A] The text contains an unresolved placeholder: 'With the rise of powerful open-source LLMs such as BLOOM (cite appropriately) and LLaMA' should be replaced with a proper citation.
  2. [Table 1] The row label 'Instuct-MT-NN' should read 'Instruct-MT-NN', and the model name is written both as Aya23-8B and Aya23-8b; please standardize.
  3. [Section 3.4] There is a typo ('Altough'), and the benchmark name is spelled inconsistently as FLoRes and FLORES; please unify.
  4. [Section 3.4] The xCOMET-10.7B metric is used without a version identifier or citation; because xCOMET versions change the score scale, this should be specified for reproducibility.
  5. [Section 4.1] The frequency-based few-shot selection method relies on an unspecified 'large Lebanese corpus' and an unreported frequency threshold; these details should be given for reproducibility.
  6. [Abstract and Section 3.1] The abstract states that datasets and code are available on GitHub, but no repository URL is provided; the GitHub release of LebEval and LW should be clearly linked.
  7. [Section 2.3 and Section 7] No manual validation or quality checks are reported for the grammar instructions synthesized by Claude 3.5 Sonnet; since Section 7 acknowledges this limitation, a brief description of any validation performed would be helpful.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's claims are empirical comparisons rather than derived predictions, and no result is shown to be equivalent to its own input by construction.

full rationale

The paper's central claims are empirical findings from fine-tuning and evaluation, not equations or first-principles derivations. There is no parameter fitted to a subset of data and then renamed as a prediction, and no benchmark is defined in terms of the claim it is used to support. The self-citations to Yakhni and Chehab (2025) for the LW dataset and for the choice of xCOMET are resources and externally falsifiable correlation evidence, not unverified uniqueness theorems, and the present fine-tuning comparisons are new experiments. A substantial validity concern remains, but it is not circularity: LebEval (Section 3.4) is constructed by the authors from a Lebanese podcast with no stated disjointness or overlap analysis relative to the LW training set (Section 3.1), and the LW-vs-NN advantage is much clearer on LebEval than on the external FLoRes benchmark. This is a benchmark-independence and possible data-leakage risk, not a derivation that reduces to its own inputs. Therefore, under the strict circularity standard, there are no circular steps to report and the score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The core empirical claim rests on three unverified assumptions: the validity of the reference-free metric for this dialect, the independence of the new evaluation benchmark from the training data, and the authenticity of the datasets. None of these is independently established in the manuscript.

free parameters (2)
  • Few-shot example count K = 3
    Selected based on performance on the evaluation sets in Section 4.1; K=5 and K=7 were rejected using the same test data, meaning test-set information influenced the final experimental configuration.
  • Curriculum data split ratio = 50/50
    For multi-step curricula, the instruction datasets were split 50% for translation instructions and 50% for contrastive instructions, a hand-chosen balance with no ablation supporting it.
assumptions (4)
  • domain assumption xCOMET-10.7B reference-free metric is a valid proxy for human-judged translation quality for Lebanese.
    Invoked in Section 3.4 as the sole evaluation metric; the supporting evidence is the authors' own prior paper, not a human evaluation conducted here.
  • domain assumption LebEval is an unbiased, independent test set for Lebanese translation.
    Section 3.4 constructs LebEval from a Lebanese podcast; the paper does not establish non-overlap with the LW training data or control for domain match, so its independence is assumed.
  • domain assumption The LW dataset is culturally authentic and suitable for training translation models.
    Taken from the authors' prior work without independent validation or inter-annotator agreement.
  • ad hoc to paper Synthetic grammar instructions from Claude 3.5 Sonnet are linguistically correct and useful for translation.
    Section 2.3 describes using Claude to generate examples from a grammar book; no human expert verification is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fine-Tuning LLMs for Low-Resource Dialect Translation: The Case of Lebanese." pith.science (2026). https://pith.science/paper/XU4523D2

@misc{pith2026250500114,
  author       = {Pith},
  title        = {Pith review of: Fine-Tuning LLMs for Low-Resource Dialect Translation: The Case of Lebanese},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XU4523D2}},
  note         = {Machine review of arXiv:2505.00114}
}
read the original abstract

This paper examines the effectiveness of Large Language Models (LLMs) in translating the low-resource Lebanese dialect, focusing on the impact of culturally authentic data versus larger translated datasets. We compare three fine-tuning approaches: Basic, contrastive, and grammar-hint tuning, using open-source Aya23 models. Experiments reveal that models fine-tuned on a smaller but culturally aware Lebanese dataset (LW) consistently outperform those trained on larger, non-native data. The best results were achieved through contrastive fine-tuning paired with contrastive prompting, which indicates the benefits of exposing translation models to bad examples. In addition, to ensure authentic evaluation, we introduce LebEval, a new benchmark derived from native Lebanese content, and compare it to the existing FLoRes benchmark. Our findings challenge the "More Data is Better" paradigm and emphasize the crucial role of cultural authenticity in dialectal translation. We made our datasets and code available on Github.

Figures

Figures reproduced from arXiv: 2505.00114 by the authors.

Figure 1
Figure 1. Example of the translation of a cultural [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Translation Instructions Templates translations. We achieve this by introducing a hint field in the training data, explicitly indicating the relevant grammatical or vocabulary rule, thereby encouraging reasoning prior to translation. Given time and resource constraints, we made an attempt to synthesize this dataset. Data Synthesis. Given a grammatical Lebanese chapter with a set of rules accompanied by illustrative … view at source ↗
Figure 3
Figure 3. Illustration of four single-step and Curriculum [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Impact of few-shot example selection meth [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Steps performed to synthesize the Lebanese Grammatical Data: 1) Choosing a Lebanese Grammatical [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Qualitative Examples that show the superiority of adapters fine-tuned using Contrastive instructions. [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 18 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Chowdhury, Maram Hasanain, Basel Mousi, Sabri Boughorbel, Yassine El Kheir, Daniel Izham, Fahim Dalvi, Majd Hawasly, Nizi Nazar, Yousseif Elshahawy, Ahmed M

    Ahmed Abdelali, Hamdy Mubarak, Shammur A. Chowdhury, Maram Hasanain, Basel Mousi, Sabri Boughorbel, Yassine El Kheir, Daniel Izham, Fahim Dalvi, Majd Hawasly, Nizi Nazar, Yousseif Elshahawy, Ahmed M. Ali, Nadir Durrani, Natasa Milic-Frayling, and Firoj Alam. 2023. https://api.semanticscholar.org/CorpusID:258865657 Larabench: Benchmarking arabic ai with la...

    work page 2023

  4. [4]

    Wael Abid. 2020. https://api.semanticscholar.org/CorpusID:227230837 The sadid evaluation datasets for low-resource spoken language machine translation of arabic dialects . In International Conference on Computational Linguistics

    work page 2020

  5. [5]

    Fakhraddin Alwajih, Gagan Bhatia, and Muhammad Abdul-Mageed. 2024. https://api.semanticscholar.org/CorpusID:271432107 Dallah: A dialect-aware multimodal large language model for arabic . ArXiv, abs/2407.18129

    work page Pith review arXiv 2024

  6. [6]

    Viraat Aryabumi, John Dang, Dwarak Talupuru, Saurabh Dash, David Cairuz, Hangyu Lin, Bharat Venkitesh, Madeline Smith, Jon Ander Campos, Yi Chern Tan, Kelly Marchisio, Max Bartolo, Sebastian Ruder, Acyr Locatelli, Julia Kreutzer, Nick Frosst, Aidan Gomez, Phil Blunsom, Marzieh Fadaee, Ahmet Üstün, and Sara Hooker. 2024. https://arxiv.org/abs/2405.15032 Ay...

    arXiv 2024

  7. [7]

    Saikat Barua. 2024. https://api.semanticscholar.org/CorpusID:269005050 Exploring autonomous agents through the lens of large language models: A review . ArXiv, abs/2404.04442

    arXiv 2024

  8. [8]

    Houda Bouamor, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdulrahim, Ossama Obeid, Salam Khalifa, Fadhl Eryani, Alexander Erdmann, and Kemal Oflazer. 2018. https://api.semanticscholar.org/CorpusID:21720875 The madar arabic dialect corpus and lexicon . In International Conference on Language Resources and Evaluation

    work page 2018

Show all 42 references
  1. [9]

    Zhaopeng Feng, Yan Zhang, Hao Li, Wenqiang Liu, Jun Lang, Yang Feng, Jian Wu, and Zuozhu Liu. 2024. https://api.semanticscholar.org/CorpusID:268032991 Improving llm-based machine translation with systematic self-correction . ArXiv, abs/2402.16379

    2024 arXiv

  2. [10]

    Patrick Fernandes, Daniel Deutsch, Mara Finkelstein, Parker Riley, Andr \'e F. T. Martins, Graham Neubig, Ankush Garg, J. Clark, Markus Freitag, and Orhan Firat. 2023. https://api.semanticscholar.org/CorpusID:260886800 The devil is in the errors: Leveraging large language mode...

    2023

  3. [11]

    Foster, Maxim Krikun, Fan Feng, Melvin Johnson, and Orhan Firat

    Xavier Garc \'i a, Yamini Bansal, Colin Cherry, George F. Foster, Maxim Krikun, Fan Feng, Melvin Johnson, and Orhan Firat. 2023. https://api.semanticscholar.org/CorpusID:256598283 The unreasonable effectiveness of few-shot learning for machine translation . ArXiv, abs/2302.01398

    2023 arXiv

  4. [12]

    Marjan Ghazvininejad, Hila Gonen, and Luke Zettlemoyer. 2023. https://api.semanticscholar.org/CorpusID:256868348 Dictionary-based phrase-level prompting of large language models for machine translation . ArXiv, abs/2302.07856

    2023 arXiv

  5. [13]

    Zhiwei He, Tian Liang, Wenxiang Jiao, Zhuosheng Zhang, Yujiu Yang, Rui Wang, Zhaopeng Tu, Shuming Shi, and Xing Wang. 2023. https://api.semanticscholar.org/CorpusID:258558158 Exploring human-like translation strategy with large language models . Transactions of the Association...

    2023

  6. [14]

    Amr Hendy, Mohamed Gomaa Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matsushita, Young Jin Kim, Mohamed Afify, and Hany Hassan Awadalla. 2023. https://api.semanticscholar.org/CorpusID:257038384 How good are gpt models at machine translation? a comprehensive ev...

    2023 arXiv

  7. [15]

    Huang Huang, Fei Yu, Jianqing Zhu, Xuening Sun, Hao Cheng, Song Dingjie, Zhihong Chen, Mosen Alharthi, Bang An, Juncai He, Ziche Liu, Junying Chen, Jianquan Li, Benyou Wang, Lian Zhang, Ruoyu Sun, Xiang Wan, Haizhou Li, and Jinchao Xu. 2024. https://doi.org/10.18653/v1/2024.na...

    2024 doi

  8. [16]

    Vivek Iyer, Pinzhen Chen, and Alexandra Birch. 2023. https://api.semanticscholar.org/CorpusID:262083660 Towards effective disambiguation for machine translation with large language models . In Conference on Machine Translation

    2023

  9. [17]

    Wenxiang Jiao, Jen tse Huang, Wenxuan Wang, Zhiwei He, Xing Wang, Shuming Shi, and Zhaopeng Tu. 2023 a . https://api.semanticscholar.org/CorpusID:257952594 Parrot: Translating during chat using large language models . ArXiv, abs/2304.02426

    2023 arXiv

  10. [18]

    Wenxiang Jiao, Wenxuan Wang, Jen tse Huang, Xing Wang, and Zhaopeng Tu. 2023 b . https://api.semanticscholar.org/CorpusID:257631519 Is chatgpt a good translator? yes with gpt-4 as the engine

    2023

  11. [19]

    Magdy, Abdul Waheed, Md Tawkat Islam Khondaker, Ahmed Oumar El-Shangiti, El Moatez Billah Nagoudi, and Muhammad Abdul-Mageed

    Karima Kadaoui, Samar M. Magdy, Abdul Waheed, Md Tawkat Islam Khondaker, Ahmed Oumar El-Shangiti, El Moatez Billah Nagoudi, and Muhammad Abdul-Mageed. 2023. https://arxiv.org/abs/2308.03051 Tarjamat: Evaluation of bard and chatgpt on machine translation of ten arabic varieties...

    2023 arXiv

  12. [20]

    Tawkat Islam Khondaker, Numaan Naeem, Fatimah Khan, AbdelRahim Elmadany, and Muhammad Abdul-Mageed

    Md. Tawkat Islam Khondaker, Numaan Naeem, Fatimah Khan, AbdelRahim Elmadany, and Muhammad Abdul-Mageed. 2024. https://api.semanticscholar.org/CorpusID:271769387 Benchmarking llama-3 on arabic language generation tasks . In ARABICNLP

    2024

  13. [21]

    Richard A. Kline. 2022. The Fundamentals of Lebanese Grammar. Taylor and Francis Ltd, London

    2022

  14. [22]

    Mateusz Krubiński, Hashem Sellat, Shadi Saleh, Adam Pospil, Petr Zemnek, and Pavel Pecina. 2023. https://api.semanticscholar.org/CorpusID:265607924 Multi-parallel corpus of north levantine arabic . In ARABICNLP

    2023

  15. [23]

    Jiahuan Li, Hao Zhou, Shujian Huang, Shan Chen, and Jiajun Chen. 2023. https://api.semanticscholar.org/CorpusID:258865882 Eliciting the translation ability of large language models via multilingual finetuning with translation instructions . Transactions of the Association for ...

    2023

  16. [24]

    Zhuoyuan Mao and Yen Yu. 2024. https://api.semanticscholar.org/CorpusID:266933003 Tuning llms with contrastive alignment instructions for machine translation in unseen, low-resource languages . ArXiv, abs/2401.05811

    2024 arXiv

  17. [25]

    Arid Hasan, Maram Hasanain, Tameem Kabbani, Fahim Dalvi, Shammur A

    Basel Mousi, Nadir Durrani, Fatema Ahmad, Md. Arid Hasan, Maram Hasanain, Tameem Kabbani, Fahim Dalvi, Shammur A. Chowdhury, and Firoj Alam. 2024. https://api.semanticscholar.org/CorpusID:272693942 Aradice: Benchmarks for dialectal and cultural capabilities in llms . ArXiv, ab...

    2024 arXiv

  18. [26]

    Vikt \'o ria Ondrejov \'a and Marek S uppa. 2024. https://doi.org/10.18653/v1/2024.vardial-1.11 Can LLM s handle low-resource dialects? a case study on translation and common sense reasoning in s ari s . In Proceedings of the Eleventh Workshop on NLP for Similar Languages, Var...

    2024 doi

  19. [27]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. https://api.semanticscholar.org/CorpusID:11080756 Bleu: a method for automatic evaluation of machine translation . In Annual Meeting of the Association for Computational Linguistics

    2002

  20. [28]

    Woodruff, Barnabas Poczos, and Hany Hassan Awadalla

    Hai Pham, Young Jin Kim, Subhabrata Mukherjee, David P. Woodruff, Barnabas Poczos, and Hany Hassan Awadalla. 2023. https://arxiv.org/abs/2308.15772 Task-based moe for multitask multilingual machine translation . Preprint, arXiv:2308.15772

    2023 arXiv

  21. [29]

    Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ili'c, Daniel Hesslow, Roman Castagn'e, Alexandra Sasha Luccioni, François Yvon, Matthias Gall \'e , Jonathan Tow, Alexander M. Rush, Stella Biderman, Albert Webson, Pawan Sasanka Ammanamanchi, Thomas Wang, Be...

    2022 arXiv

  22. [30]

    Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Satheesh Katipomu, Haonan Li, Fajri Koto, William Marshall, Gurpreet Gosal, Cynthia Liu, Zhiming Chen, Osama Mohammed Afzal, Samta Kamboj, Onkar Pandit, Rahul Pal, Lalit Pradhan, Zain Muhammad Mujahid, Massa Baali, Xudong Han, Sondo...

    2023 arXiv

  23. [31]

    Jiho Shin, Clark Tang, Tahmineh Mohati, Maleknaz Nayebi, Song Wang, and Hadi Hemmati. 2023. https://arxiv.org/abs/2310.10508 Prompt engineering or fine tuning: An empirical assessment of large language models in automated software engineering tasks . Preprint, arXiv:2310.10508

    2023 arXiv

  24. [32]

    Garrett Tanzer, Mirac Suzgun, Eline Visser, Dan Jurafsky, and Luke Melas-Kyriazi. 2023. https://api.semanticscholar.org/CorpusID:263134958 A benchmark for learning to translate a new language from one grammar book . ArXiv, abs/2309.16575

    2023 arXiv

  25. [33]

    Hashimoto

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca

    2023

  26. [34]

    Spruit, C

    Nllb team, Marta Ruiz Costa-juss \`a , James Cross, Onur cCelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Alison Youngblood, Bapi Akula, Lo \"i c Barrault, Gabriel Mejia ...

    2022 arXiv

  27. [35]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. https://api.semanticscholar.org...

    2023 arXiv

  28. [36]

    Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. 2024 a . https://api.semanticscholar.org/CorpusID:271745981 Wizardlm: Empowering large pre-trained language models to follow complex instructions . In International...

    2024

  29. [37]

    Haoran Xu, Amr Sharaf, Yunmo Chen, Weiting Tan, Lingfeng Shen, Benjamin Van Durme, Kenton Murray, and Young Jin Kim. 2024 b . https://api.semanticscholar.org/CorpusID:267028540 Contrastive preference optimization: Pushing the boundaries of llm performance in machine translatio...

    2024 arXiv

  30. [38]

    Silvana Yakhni and Ali Chehab. 2025. https://aclanthology.org/2025.abjadnlp-1.13/ Can LLM s translate cultural nuance in dialects? a case study on L ebanese A rabic . In Proceedings of the 1st Workshop on NLP for Languages Using Arabic Script, pages 114--135, Abu Dhabi, UAE. A...

    2025

  31. [39]

    Binwei Yao, Ming Jiang, Tara Bobinac, Diyi Yang, and Junjie Hu. 2024. https://aclanthology.org/2024.findings-emnlp.765 Benchmarking machine translation with cultural awareness . In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 13078--13096, Miami...

    2024

  32. [40]

    Biao Zhang, Barry Haddow, and Alexandra Birch. 2023 a . https://api.semanticscholar.org/CorpusID:255942578 Prompting large language model for machine translation: A case study . ArXiv, abs/2301.07069

    2023 arXiv

  33. [41]

    Chen Zhang, Xiao Liu, Jiuheng Lin, and Yansong Feng. 2024. https://api.semanticscholar.org/CorpusID:268063652 Teaching large language models an unseen language on the fly . ArXiv, abs/2402.19167

    2024 arXiv

  34. [42]

    Yu, Mengzhao Jia, Meng Jiang, and Francesco Barbieri

    Zhihan Zhang, Dong-Ho Lee, Yuwei Fang, W. Yu, Mengzhao Jia, Meng Jiang, and Francesco Barbieri. 2023 b . https://api.semanticscholar.org/CorpusID:265212710 Plug: Leveraging pivot language in cross-lingual instruction tuning . ArXiv, abs/2311.08711

    2023 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.