Pith. sign in

REVIEW 3 major objections 5 minor 37 references

Building Large-Scale English-Romanian Literary Translation Resources with Open Models

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A 12-billion-parameter open model, fine-tuned on 15,000 synthetic fable translations, scores 4.83/5 on a literary-quality rubric — 0.09 short of GPT-o3 — while translating three million fables for about $350.

desk verdict The released 15K/3M EN-RO fable corpus and fine-tuned models are a real resource for low-resource Romanian literary MT, but the headline near-parity claim rests on a 100-sample, same-family LLM judge with no human anchor and no error bars. read the letter →

arxiv 2509.07829 v4 pith:T5O44U5S submitted 2025-09-09 cs.CL cs.AIcs.LG

Mihai Nadas , Laura Diosan , Andreea Tomescu , Andrei Piscoran This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords literarymachinetranslationlow-resourcelanguagesEnglish-Romaniansyntheticparalleldatamoralfablesparameter-efficientfine-tuningLoRALLM-as-a-judgeevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that small, open, fine-tuned models can translate literary text into a low-resource language almost as well as the biggest proprietary systems, and at a tiny fraction of the cost. Concretely, it builds the TinyFabulist Translation Framework (TF2): using 15,000 GPT-o3-generated English–Romanian fable pairs as silver-standard references, it fine-tunes Gemma-3 open models of 1B, 4B, and 12B parameters with LoRA adapters, then uses the best of them to translate all three million fables of the synthetic TinyFabulist corpus into Romanian. On a five-dimension rubric (accuracy, fluency, coherence, style, cultural adaptation) judged by GPT-o3-mini, the best model, TF2-12B, scores 4.83 out of 5 at greedy decoding — within 0.09 of GPT-o3's 4.92 and well above its untuned base of 4.43. The paper estimates that translating the full corpus with the open model costs about $350 in rented GPUs, versus $2,700 to $270,000 for commercial APIs. The stakes are practical: if the claim holds, high-quality literary translation into underserved languages stops being a privilege reserved for teams with large API budgets.

What carries the argument

The load-bearing mechanism is the four-stage TinyFabulist Translation Framework (TF2) pipeline. Stage 1 benchmarks 13 systems with a five-dimension LLM-as-judge rubric and designates the top scorer (GPT-o3) as reference translator and the top open model (Gemma-3-12B-it) as backbone. Stage 2 produces the 15,000-pair silver-standard parallel corpus with GPT-o3. Stage 3 applies Low-Rank Adaptation (LoRA), learning a low-rank update $\Delta W = \frac{\alpha}{r} B A$ with rank $r = 32$, $\alpha = 32$, and dropout 0.05 on the attention and feed-forward projections, then merges the adapter into the frozen weights. Stage 4 runs the fine-tuned model over the three-million-fable source corpus at greedy decoding ($T = 0.0$), the temperature the authors show is optimal. The five-dimension rubric — accuracy, fluency, coherence, style, and cultural/pragmatic adaptation — is the yardstick that carries the near-parity claim; BLEU against the GPT-o3 references is the secondary consistency check, and the Grok-3-mini re-scoring of the same 100 fables is the paper's safeguard against judge-family bias.

What would settle it

Two checks settle it. First, compute paired per-fable score differences between TF2-12B and GPT-o3 on the paper's own 100-fable sample: the near-parity claim requires the 95 percent confidence interval of the mean gap to sit near 0.09, and if it lies above 0.2 the headline gap is not statistically near-parity. Second, have professional Romanian translators blindly rank the two systems' outputs on the same five dimensions: if humans place GPT-o3 more than half a point ahead on average, the automated judge was flattering the smaller model.

Watch

Extended reading notes

Core claim

The central claim, stated as the authors would state it to a fair reader, is that parameter-efficient fine-tuning on a modest amount of high-quality synthetic parallel data brings an open 12B-parameter model to near-parity with the strongest proprietary systems on literary translation. TF2-12B — Gemma-3-12B-it adapted with LoRA on 12,000 training pairs from the GPT-o3-generated DS-TF2-EN-RO-15K set — reaches an average rubric score of 4.83 and BLEU 0.0926 (9.26 on the conventional scale) at greedy decoding, against GPT-o3's 4.92; a cross-family judge, Grok-3-mini, reproduces the ranking with an even smaller gap of 0.02 to 0.07. The paper also claims the recipe transfers across scales (the 4B model reaches 4.74, the 1B model jumps from 2.02 to 3.75), that 8-bit quantization is nearly lossless, and that the resulting three-million-pair corpus, DS-TF2-EN-RO-3M, is the largest openly available English–Romanian literary dataset.

Load-bearing premise

The load-bearing premise is that the automated judge's scores — five rubric dimensions averaged over just 100 fables, with no human anchor and no confidence intervals — measure literary translation quality precisely enough that a 0.09-point gap counts as near-parity.

Editorial extensions

If this is right

  • Literary translation into Romanian at corpus scale becomes affordable: the paper's estimate is roughly $350 of rented GPU time for three million fables, against $2,700 to $270,000 through commercial APIs.
  • The openly released 15K and 3M parallel fable datasets give researchers the first large-scale English–Romanian literary corpus, with full provenance metadata for reproducibility.
  • The recipe is size-flexible: TF2-4B scores 4.74 and even the 1B model improves about 86 percent over its untuned base, so teams can trade quality against hardware budget.
  • The ablations give concrete deployment guidance: decode at $T = 0.0$ and use the 8-bit quantized checkpoints, which track their FP16 counterparts within 0.01 to 0.03 rubric points.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Per-dimension scores are more uneven than the headline average suggests: TF2-12B's largest deficit is accuracy (4.72 against GPT-o3's 4.86) and its smallest is fluency (4.88 against 4.92). My reading is that the near-parity is most secure on style and naturalness and least secure on faithful meaning transfer.
  • Because the reference translator and the primary judge come from the same model family, the reported 0.09-point gap measures closeness to GPT-o3's renderings, not to human literary translation; treating it as the true quality difference is a working bet until a human gold set exists.
  • The cheapest decisive extension is to clone the pipeline for another low-resource pair with a different reference translator and a different judge family; if the fine-tuned open model's accuracy gap stays near 0.14 points there too, the near-parity pattern is a property of the method rather than of the specific judge.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces TinyFabulist Translation Framework (TF2), an end-to-end pipeline for English-to-Romanian literary translation of moral fables. The pipeline (S1–S4) benchmarks candidate translators, selects GPT-o3 to generate 15K silver-standard Romanian references (DS-TF2-EN-RO-15K), fine-tunes Gemma-3 1B/4B/12B backbones with LoRA, and uses the fine-tuned models to translate the 3M-fable DS-TF1-EN-3M corpus into Romanian (DS-TF2-EN-RO-3M). Evaluation combines BLEU with a five-dimension LLM rubric scored by GPT-o3-mini. The central claim is that TF2-12B achieves near-parity with GPT-o3 (average rubric 4.83 vs. 4.92) at roughly $350 of compute, and the paper releases models, datasets, prompts, and scripts.

Significance. If the central comparative claim were established, the paper would make a useful contribution to low-resource literary MT: it provides a concrete, reproducible pipeline, releases open models and two large parallel corpora, reports hardware and cost details transparently, and makes a good-faith attempt to check judge-family bias via a cross-family judge. The paper is also unusually candid in Section 7 about the limitations of LLM-based evaluation and synthetic references, which supports reproducibility and honest interpretation. However, the headline 'near-parity' claim is currently supported only by a 100-item LLM-judge evaluation with no human anchor, no confidence intervals, and no significance test, and the judge and references come from the same model family. These weaknesses are load-bearing for the paper's main contribution, so the result, while promising, is not yet established to the standard required for the claims made in the abstract and title.

major comments (3)
  1. [Section 4.2, Table 3; Section 7] The headline near-parity claim (TF2-12B 4.83 vs. GPT-o3 4.92 at T=0.0) rests on rubric scores from a single LLM judge over 100 fables. The paper reports no confidence intervals, no significance test, and no pre-specified parity threshold. Section 7 explicitly notes that the 100-item subset is 2% of the test set and that 'automatic LLM-based scoring remains a proxy.' Because all scores cluster between 4.8 and 4.9 on a 1–5 scale, a 0.09-point gap is within plausible judge noise, and the near-parity claim is not statistically supported. This is load-bearing for the central conclusion and needs either a human evaluation on a representative sample, per-item variance and significance testing, or a substantial tempering of the claim.
  2. [Sections 3.1, 3.2, 4.2.1, Table 4] The evaluation pipeline is circular in a way that the cross-family check does not resolve. The silver-standard references are GPT-o3 outputs (Section 3.2), the BLEU references are GPT-o3 outputs (Table 2 note), and the primary judge is GPT-o3-mini (Table 1). The cross-family check in Section 4.2.1 only replaces the judge with Grok-3-mini while keeping the GPT-o3 references, so it cannot detect bias in the reference set itself. Table 4 reports exactly the same 4.92 score for GPT-o3 under both judges, which suggests the judges may be anchoring on reference identity rather than independently assessing quality; at minimum, the identical score warrants discussion. A valid robustness check would need references from a different model family or from human translators, and would report per-item score distributions rather than single averages.
  3. [Abstract; Section 7] The abstract states that the fine-tuned model narrows the gap to proprietary models 'under automated and human-anchored evaluation,' but the manuscript contains no human evaluation whatsoever. Section 7 concedes that 'automatic LLM-based scoring remains a proxy' and that the benchmark measures similarity to a machine translation rather than to human translation. This is not a peripheral presentation issue: the phrase 'human-anchored evaluation' overstates the evidence and should be removed unless a human study is added. The paper's own limitation statements in Sections 7 directly undercut the title's 'near-parity' claim as currently formulated.
minor comments (5)
  1. [Title (full text)] The title in the full text is rendered as 'SMALLOPENMODELSACHIEVENEAR-PARITY WITHLARGE MODELS INLOW-RESOURCELITERARYTRANSLATION AT A FRACTION OF THECOST' with missing spaces; it should read 'Small Open Models Achieve Near-Parity with Large Models in Low-Resource Literary Translation at a Fraction of the Cost.'
  2. [Section 1.1] The research question RQ1 is listed twice ('RQ1Cost-Constrained Data Generation:' appears twice); the duplicate should be removed.
  3. [Section 4.2.1] The text says the cross-family check used 'the same rubric, prompt, and randomized system order defined in Section 3.1,' but Section 3.1 does not describe randomization of system order; please clarify the protocol or remove the reference.
  4. [Table 2 note / Section 4.2] The BLEU results are confusing as reported: the smallest model (TF2-1B-distilled) has the highest BLEU (0.2180), while TF2-12B has 0.0926. The explanation that larger models paraphrase more is plausible, but it should appear with the table or in the main text rather than only in a brief note, otherwise readers may misinterpret the metric.
  5. [Section 3.3(d)] The citation [12] (G-Eval) is used to support early stopping based on validation loss, but G-Eval is an evaluation framework, not a training recipe; a more appropriate citation for early stopping or loss-based model selection should be given.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the near-parity claim is an empirical evaluation result with acknowledged same-family-judge limitations, not a derivation whose output is fixed by its inputs.

full rationale

The central claim is a comparative evaluation result: TF2-12B scores 4.83 vs GPT-o3's 4.92 on a five-dimension LLM rubric and BLEU 0.0926 against GPT-o3 references. This is an experimental observation, not a quantity that is equivalent to its inputs by construction. The pipeline does use the GPT-o3 family in three roles—GPT-o3 generates the 15k silver-standard references, GPT-o3-mini serves as the primary LLM judge, and GPT-o3 is the proprietary comparison target—which creates a real risk of same-family favoritism. The paper explicitly acknowledges this in Section 7: 'our silver-standard references are themselves generated by a GPT-o3 variant, which raises the possibility of family favoritism.' It then runs a cross-family judge check with Grok-3-mini in Section 4.2.1 (Table 4), which reports a stable ranking and a smaller TF2–o3 gap under the alternative judge. That check does not eliminate all proxy concerns, but it means the result is not forced by definition. None of the enumerated circularity patterns is present: no parameter is fitted to the evaluation scores and then renamed as a prediction; no equation defines the reported scores in terms of the training references; the self-citations to DS-TF1-EN-3M [18] are dataset provenance rather than load-bearing evidence for the parity claim; and no uniqueness theorem or ansatz is imported from the authors' prior work. The abstract's phrase 'human-anchored evaluation' is unsupported by any human evaluation in the manuscript, and the lack of confidence intervals or a pre-specified parity threshold is a validity concern, but under the stated rules these are correctness risks rather than circularity. The derivation chain is therefore self-contained in the sense required by this analysis.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the validity of the LLM-judge loop and on the quality of synthetic references; the released artifacts themselves do not depend on theoretical postulates.

free parameters (5)
  • Decoding temperature (headline) = 0.0
    Selected on the held-out test set from {0.0, 0.2, 0.5, 1.0}; the best-scoring value is reported as the headline and used for the 3M corpus run.
  • LoRA rank r and alpha = r=32, alpha=32
    Chosen by hand following common practice; these control adapter capacity and affect all reported rubric and BLEU scores.
  • LoRA dropout = 0.05
    Set by hand; affects regularization and the quality of the adapted models.
  • Number of silver seed translations = 15,000
    A budget-driven choice; the size and selection of this seed determine training signal and costs.
  • Thinking-token ratio for cost estimates = 1.0 (medium reasoning)
    Assumes GPT-o3 hidden reasoning tokens equal visible output tokens; the paper's sensitivity analysis shows costs vary 3-fold across low/high settings.
assumptions (4)
  • domain assumption GPT-o3-mini rubric scores are a valid proxy for human judgment of literary translation quality
    All comparative conclusions use this judge; the paper flags it as a proxy in Section 7 but provides no human validation.
  • domain assumption GPT-o3 translations are high-quality silver references for training and BLEU scoring
    Used as training targets and as BLEU references (Section 3.2); the paper acknowledges these may contain subtle errors.
  • domain assumption Moral fables are a representative literary genre for the claimed results
    The paper warns in Section 7 that formulaic fables may inflate quality estimates and that results may not generalize.
  • domain assumption Published API prices and GPU rental rates remained valid for the study period
    The cost comparison in Table 5 and Appendix A depends on these rates, which the paper notes are time-sensitive.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Building Large-Scale English-Romanian Literary Translation Resources with Open Models." pith.science (2026). https://pith.science/paper/T5O44U5S

@misc{pith2026250907829,
  author       = {Pith},
  title        = {Pith review of: Building Large-Scale English-Romanian Literary Translation Resources with Open Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/T5O44U5S}},
  note         = {Machine review of arXiv:2509.07829}
}
abstract

Literary translation has recently gained attention as a distinct and complex task in machine translation research, yet translation by small open models remains an open problem, particularly for low-resource languages such as Romanian. We introduce the TinyFabulist Translation Framework (TF2), a unified framework for dataset creation, fine-tuning, and evaluation in English $\to$ Romanian literary translation. Building on DS-TF1-EN-3M, the largest collection of synthetic English fables to date, our pipeline first generates 15k high-quality Romanian references from the TF1 pool using a high-performing large language model (LLM). We then apply a two-stage fine-tuning process to a 12B-parameter open-weight model: (i) instruction tuning to capture genre-specific narrative style, and (ii) adapter compression for efficient deployment. Evaluation combines a five-dimension LLM-based rubric (accuracy, fluency, coherence, style, cultural adaptation) as the primary comparative framework, alongside corpus-level Bilingual Evaluation Understudy (BLEU) reported as a secondary reference-based consistency metric. Our fine-tuned model (TF2-12B) achieves strong fluency and adequacy, narrowing the gap to top-performing proprietary models under automated and human-anchored evaluation, while being open, accessible, and significantly more cost-effective. We publicly release the fine-tuned model and two large-scale synthetic parallel datasets (DS-TF2-EN-RO-3M and DS-TF2-EN-RO-15K), along with all scripts and evaluation prompts. TF2 provides an end-to-end, reproducible pipeline for research on cost-efficient translation, cross-lingual narrative generation, and the broad adoption of open models for culturally significant literary content in low-resource settings.

Figures

Figures reproduced from arXiv: 2509.07829 by the authors.

Figure 1
Figure 1. TINYFABULIST TRANSLATION FRAMEWORK (TF2) pipeline: Evaluation of translation models on literary benchmarks (S1); creation of a 15k English–Romanian parallel corpus via best system (S2); parameter-efficient fine￾tuning of open LLMs and quantized variants (S3); and large-scale Romanian translation of the full English corpus (S4). 3.1 Stage 1: Evaluating Models & Selecting Translators The first stage identifies (i) a h… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 9 canonical work pages

  1. [1]

    DeepSeek-AI, Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, Huazuo Gao, Kaige Gao, Wenjun Gao, Ruiqi Ge, Kang Guan, Daya Guo, Jianzhong Guo, Guangbo Hao, Zhewen Hao, Ying He, Wenjie Hu, Panpan Huang, Erhang Li, Guowei Li, Jiashi Li, Yao Li, Y . K. Li, Wenfeng Liang, Fangyun Lin, A. X....

    arXiv 2024

  2. [2]

    GPT3.int8(): 8-bit Matrix Multiplication for Transformers at Scale.Advances in Neural Information Processing Systems, 35: 30318–30332, December 2022

    Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. GPT3.int8(): 8-bit Matrix Multiplication for Transformers at Scale.Advances in Neural Information Processing Systems, 35: 30318–30332, December 2022. URL https://proceedings.neurips.cc/paper_files/paper/2022/hash/ c3ba4962c05c49636d4c6206a97e9c8a-Abstract-Conference.html

    work page 2022

  3. [3]

    LoRA-C: Parameter- Efficient Fine-Tuning of Robust CNN for IoT Devices, November 2024

    Chuntao Ding, Xu Cao, Jianhang Xie, Linlin Fan, Shangguang Wang, and Zhichao Lu. LoRA-C: Parameter- Efficient Fine-Tuning of Robust CNN for IoT Devices, November 2024. URL http://arxiv.org/abs/2410. 16954. arXiv:2410.16954 [cs]

    arXiv 2024

  4. [4]

    TinyStories: How Small Can Language Models Be and Still Speak Coherent English?, May 2023

    Ronen Eldan and Yuanzhi Li. TinyStories: How Small Can Language Models Be and Still Speak Coherent English?, May 2023. URLhttp://arxiv.org/abs/2305.07759. arXiv:2305.07759 [cs]

    arXiv 2023

  5. [5]

    GPTQ: Accurate Post-Training Quan- tization for Generative Pre-trained Transformers, March 2023

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. GPTQ: Accurate Post-Training Quan- tization for Generative Pre-trained Transformers, March 2023. URL http://arxiv.org/abs/2210.17323. arXiv:2210.17323 [cs]

    arXiv 2023

  6. [6]

    A Corpus for Understanding and Generating Moral Stories, April 2022

    Jian Guan, Ziqi Liu, and Minlie Huang. A Corpus for Understanding and Generating Moral Stories, April 2022. URLhttp://arxiv.org/abs/2204.09438. arXiv:2204.09438 [cs]

    arXiv 2022

  7. [7]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-Rank Adaptation of Large Language Models, October 2021. URL http://arxiv.org/abs/ 2106.09685. arXiv:2106.09685 [cs]. 16 APREPRINT- SEPTEMBER15, 2025

    arXiv 2021

  8. [8]

    Exploring Context-Aware Evaluation Metrics for Machine Translation

    Xinyu Hu, Xunjian Yin, and Xiaojun Wan. Exploring Context-Aware Evaluation Metrics for Machine Translation. In Houda Bouamor, Juan Pino, and Kalika Bali, editors,Findings of the Association for Computational Linguistics: EMNLP 2023, pages 15291–15298, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.findings-emnlp...

    doi:10.18653/v1/2023.findings-emnlp.1021 2023

Show all 37 references
  1. [9]

    Europarl: A Parallel Corpus for Statistical Machine Translation

    Philipp Koehn. Europarl: A Parallel Corpus for Statistical Machine Translation. InThe Tenth Machine Translation Summit Proceedings of Conference, pages 79–86. International Associ- ation for Machine Translation, 2005. URL https://www.research.ed.ac.uk/en/publications/ europarl...

    2005

  2. [10]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient Memory Management for Large Language Model Serving with PagedAttention, September 2023. URLhttp://arxiv.org/abs/2309.06180. arXiv:2309.06180 [cs]

    2023 arXiv

  3. [11]

    AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration, July 2024

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration, July 2024. URLhttp://arxiv.org/abs/2306.00978. arXiv:2306.00978 [cs]

    2024 arXiv

  4. [12]

    G-Eval: NLG Eval- uation using GPT-4 with Better Human Alignment, May 2023

    Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-Eval: NLG Eval- uation using GPT-4 with Better Human Alignment, May 2023. URL http://arxiv.org/abs/2303.16634. arXiv:2303.16634 [cs]

    2023 arXiv

  5. [13]

    Multidimensional Quality Metrics (MQM) : a Framework for Declaring and Describing Translation Quality Metrics.Tradumàtica, 1(12):0455–463, 2014

    Arle Lommel, Hans Uszkoreit, and Aljoscha Burchardt. Multidimensional Quality Metrics (MQM) : a Framework for Declaring and Describing Translation Quality Metrics.Tradumàtica, 1(12):0455–463, 2014. ISSN 1578-7559. doi: 10.5565/rev/tradumatica.77. URLhttps://ddd.uab.cat/record/130144

    2014 doi

  6. [14]

    On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey, June 2024

    Lin Long, Rui Wang, Ruixuan Xiao, Junbo Zhao, Xiao Ding, Gang Chen, and Haobo Wang. On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey, June 2024. URL http://arxiv.org/abs/2406. 15126. arXiv:2406.15126 [cs]

    2024 arXiv

  7. [15]

    A survey on LoRA of large language models.Frontiers of Computer Science, 19(7):197605, December 2024

    Yuren Mao, Yuhang Ge, Yijiang Fan, Wenyi Xu, Yu Mi, Zhonghao Hu, and Yunjun Gao. A survey on LoRA of large language models.Frontiers of Computer Science, 19(7):197605, December 2024. ISSN 2095-2236. doi: 10.1007/s11704-024-40663-9. URLhttps://doi.org/10.1007/s11704-024-40663-9

    2024 doi

  8. [16]

    Guerreiro, Ricardo Rei, Amin Farajian, Mateusz Klimaszewski, Duarte M

    Pedro Henrique Martins, João Alves, Patrick Fernandes, Nuno M. Guerreiro, Ricardo Rei, Amin Farajian, Mateusz Klimaszewski, Duarte M. Alves, José Pombal, Nicolas Boizard, Manuel Faysse, Pierre Colombo, François Yvon, Barry Haddow, José G. C. de Souza, Alexandra Birch, and Andr...

    2025 arXiv

  9. [17]

    V orbe¸ sti Române¸ ste?

    Mihai Masala, Denis C. Ilie-Ablachim, Alexandru Dima, Dragos Corlatescu, Miruna Zavelca, Ovio Olaru, Simina Terian, Andrei Terian, Marius Leordeanu, Horia Velicu, Marius Popescu, Mihai Dascalu, and Traian Rebedea. "V orbe¸ sti Române¸ ste?" A Recipe to Train Powerful Romanian ...

    2024 arXiv

  10. [18]

    TF1-EN-3M: Three Million Synthetic Moral Fables for Training Small, Open Language Models, April 2025

    Mihai Nadas, Laura Diosan, Andrei Piscoran, and Andreea Tomescu. TF1-EN-3M: Three Million Synthetic Moral Fables for Training Small, Open Language Models, April 2025. URL http://arxiv.org/abs/2504.20605. arXiv:2504.20605 [cs]

    2025 arXiv

  11. [19]

    Synthetic Data Generation Using Large Language Models: Advances in Text and Code, March 2025

    Mihai Nadas, Laura Diosan, and Andreea Tomescu. Synthetic Data Generation Using Large Language Models: Advances in Text and Code, March 2025. URLhttp://arxiv.org/abs/2503.14023. arXiv:2503.14023 [cs]

    2025

  12. [20]

    Bleu: a Method for Automatic Evaluation of Machine Translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a Method for Automatic Evaluation of Machine Translation. In Pierre Isabelle, Eugene Charniak, and Dekang Lin, editors,Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pa...

    2002

  13. [21]

    Farinha, and Alon Lavie

    Ricardo Rei, Craig Stewart, Ana C. Farinha, and Alon Lavie. COMET: A Neural Framework for MT Evaluation, October 2020. URLhttp://arxiv.org/abs/2009.09025. arXiv:2009.09025 [cs]

    2020 arXiv

  14. [22]

    Improving Neural Machine Translation Models with Monolingual Data, June 2016

    Rico Sennrich, Barry Haddow, and Alexandra Birch. Improving Neural Machine Translation Models with Monolingual Data, June 2016. URLhttp://arxiv.org/abs/1511.06709. arXiv:1511.06709 [cs]

    2016 arXiv

  15. [23]

    LLM Compressor is here: Faster inference with vLLM, August 2024

    Robert Shaw and Mark Kurtz. LLM Compressor is here: Faster inference with vLLM, August 2024. URL https: //developers.redhat.com/articles/2024/08/14/llm-compressor-here-faster-inference-vllm

    2024

  16. [24]

    Benchmarking Low-Resource Machine Translation Systems

    Ana Silva, Nikit Srivastava, Tatiana Moteu Ngoli, Michael Röder, Diego Moussallem, and Axel-Cyrille Ngonga Ngomo. Benchmarking Low-Resource Machine Translation Systems. In Atul Kr. Ojha, Chao-hong 17 APREPRINT- SEPTEMBER15, 2025 Liu, Ekaterina Vylomova, Flammie Pirinen, Jade A...

    2025

  17. [25]

    BLEU is Not Suitable for the Evaluation of Text Simplification, October 2018

    Elior Sulem, Omri Abend, and Ari Rappoport. BLEU is Not Suitable for the Evaluation of Text Simplification, October 2018. URLhttp://arxiv.org/abs/1810.05995. arXiv:1810.05995 [cs]

    2018 arXiv

  18. [26]

    CLIRMatrix: A massively large collection of bilingual and multilingual datasets for Cross-Lingual Information Retrieval

    Shuo Sun and Kevin Duh. CLIRMatrix: A massively large collection of bilingual and multilingual datasets for Cross-Lingual Information Retrieval. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural L...

    2020 doi

  19. [27]

    Nllb Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Young- blood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Pra...

    2022 arXiv

  20. [28]

    Parallel Data, Tools and Interfaces in OPUS.Lrec, 2012:2214–2218, 2012

    Jorg Tiedemann. Parallel Data, Tools and Interfaces in OPUS.Lrec, 2012:2214–2218, 2012

    2012

  21. [29]

    MAATS: A Multi-Agent Automated Translation System Based on MQM Evaluation, August 2025

    George Wang, Jiaqian Hu, and Safinah Ali. MAATS: A Multi-Agent Automated Translation System Based on MQM Evaluation, August 2025. URLhttp://arxiv.org/abs/2505.14848. arXiv:2505.14848 [cs]

    2025 arXiv

  22. [30]

    Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategies, June 2024

    Junlin Wang, Siddhartha Jain, Dejiao Zhang, Baishakhi Ray, Varun Kumar, and Ben Athiwaratkun. Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategies, June 2024. URL http://arxiv. org/abs/2406.06461. arXiv:2406.06461 [cs]

    2024 arXiv

  23. [31]

    Smith, Daniel Khashabi, and Hannaneh Hajishirzi

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-Instruct: Aligning Language Models with Self-Generated Instructions, May 2023. URL http: //arxiv.org/abs/2212.10560. arXiv:2212.10560 [cs]

    2023 arXiv

  24. [32]

    Exploring Parameter-Efficient Fine- Tuning Techniques for Code Generation with Large Language Models.ACM Trans

    Martin Weyssow, Xin Zhou, Kisub Kim, David Lo, and Houari Sahraoui. Exploring Parameter-Efficient Fine- Tuning Techniques for Code Generation with Large Language Models.ACM Trans. Softw. Eng. Methodol., January 2025. ISSN 1049-331X. doi: 10.1145/3714461. URL https://dl.acm.org...

    2025 doi

  25. [33]

    mT5: A massively multilingual pre-trained text-to-text transformer, March 2021

    Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. mT5: A massively multilingual pre-trained text-to-text transformer, March 2021. URL http://arxiv.org/abs/2010.11934. arXiv:2010.11934 [cs] version: 3

    2021 arXiv

  26. [34]

    Benchmarking Machine Translation with Cultural Awareness, October 2024

    Binwei Yao, Ming Jiang, Tara Bobinac, Diyi Yang, and Junjie Hu. Benchmarking Machine Translation with Cultural Awareness, October 2024. URLhttp://arxiv.org/abs/2305.14328. arXiv:2305.14328 [cs]

    2024 arXiv

  27. [35]

    LiTransProQA: an LLM-based Literary Translation evaluation metric with Professional Question Answering, May 2025

    Ran Zhang, Wei Zhao, Lieve Macken, and Steffen Eger. LiTransProQA: an LLM-based Literary Translation evaluation metric with Professional Question Answering, May 2025. URLhttp://arxiv.org/abs/2505.05423. arXiv:2505.05423 [cs]

    2025

  28. [36]

    medium reasoning

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena, December 2023. URL http://arxiv.org/abs/2...

    2023 arXiv

  29. [340]

    URLhttps://aclanthology.org/2020.emnlp-main.340/

    2020

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.