Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Proverbs Run in Pairs: Evaluating Proverb Translation Capability of Large Language Model

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Large instruction-tuned language models generally translate proverbs better than dedicated NMT systems, and standard automatic metrics including BLEU, CHRF++, and COMET cannot reliably measure proverb translation quality.

desk verdict New benchmark with a real metric caveat, but the LLM-superiority claim outruns the evidence. read the letter →

arxiv 2501.11953 v1 pith:E7EGDWBQ submitted 2025-01-21 cs.CL

Minghan Wang , Viet-Thanh Pham , Farhad Moghimifar , Thuy-Trang Vu This is my paper · ORCID
classification cs.CL
keywords proverbtranslationlargelanguagemodelsneuralmachineautomaticevaluationmetricsfigurativeconversationcontextculturaladaptationdatasetconstruction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Proverbs are short, culturally loaded sayings, and translating them well requires more than word-for-word fidelity. This paper asks whether current machine translation systems can do that, and whether standard automatic metrics can tell us when they have. Using new datasets of standalone proverbs and proverbs embedded in movie-subtitle conversations across English, German, Bengali, Indonesian, and Chinese, the authors find that large instruction-tuned language models generally outperform dedicated NMT systems at proverb translation, especially between culturally similar languages. They also show that BLEU, CHRF++, and COMET routinely score meaning-equivalent paraphrases lower than literal matches, so these metrics are not reliable for evaluating figurative language. Conversation context, delivered in dialogue form, improves translation, while giving the model a proverb explanation does not.

What carries the argument

The central object is a pair of purpose-built evaluation resources: the standalone proverb translation set, extending the MAPS multicultural proverb collection with human-verified translations, and the Proverb-in-Conversation (PiC) set, mined from OpenSubtitles and filtered through LLM-based usage detection, quality estimation, and human review. The argument also relies on a diagnostic that pairs sentence-embedding cosine similarity with BLEU/CHRF++/COMET score gaps to identify cases where near-synonymous hypotheses are scored far apart. These datasets and the diagnostic carry the paper's case that the metrics understate the quality of creative, culturally adaptive translations.

What would settle it

A human evaluation study that has bilingual raters judge a random sample of the released translations would settle both claims: if human ranking of models differs from the BLEU/CHRF++/COMET ranking, the metric critique holds, and if humans do not rank LLMs above NMT, the LLM-superiority claim fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that state-of-the-art LLMs—particularly Llama-3.1-70B and GPT-4o-mini—are better at translating proverbs than dedicated NMT systems such as NLLB and ALMA-R, while the metrics used to measure that performance are systematically misleading for figurative language. The evidence comes from two newly built test sets: a standalone proverb translation set extending the MAPS collection, and a Proverb-in-Conversation set mined from OpenSubtitles, filtered by LLM checks, quality estimation, and human annotation. Experiments show that models translate proverbs more accurately when the source and target languages share cultural context (German–English) and that figurative proverbs are harder than literal ones across all models. The metric critique is supported by qualitative examples where a semantically faithful paraphrase scores far below a literal word-match, and by a large-scale scan showing 22,704 hypothesis pairs with near-identical semantics but large metric gaps; COMET, though more robust, still fails on metaphoric reformulation. The authors conclude that LLM-as-a-judge ratings, which align only loosely with COMET, do not resolve the problem.

Load-bearing premise

The rankings of models and language pairs in the paper assume that BLEU, CHRF++, and COMET, though unreliable for individual proverbs, are still accurate enough on average to say which model is better—an assumption the paper never tests with human judgments or significance tests.

Editorial extensions

If this is right

  • Larger LLMs already know proverb meanings from pretraining, so adding an explanation or a one-shot example does not improve their translations; conversational context in dialogue form matters more.
  • One-shot prompting generally helps smaller models and the from-English direction, but the strongest model sometimes does better zero-shot when translating into English.
  • Proverb translation quality is higher between culturally similar, high-resource languages (German–English) and lower between distant ones (Bengali–English).
  • NMT systems stay competitive on literal proverbs in conversation but fall behind LLMs on figurative proverbs, which require cultural adaptation.
  • BLEU, CHRF++, and COMET can rank a faithful paraphrase below a literal word-match, so they should not be the sole judge of figurative translation quality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the metric critique holds, earlier MT evaluations that relied on BLEU or COMET may have systematically understated LLM performance on idiomatic and proverbial language; a culturally aware metric could reorder model rankings.
  • The Proverb-in-Conversation dataset offers a testbed for designing evaluation metrics that reward cultural equivalence rather than lexical overlap, and for fine-tuning models to exploit conversational context.
  • The finding that dialogue-form context helps more than concatenated context suggests document-level MT systems should model multi-turn conversational structure explicitly.
  • Because the paper uses only five languages, an immediate extension is to test whether LLM superiority and metric failure generalize to more distant language pairs and to proverbs from oral, low-resource cultures.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper constructs two proverb-translation benchmarks: a standalone proverb dataset built by extending MAPS, and a Proverb-in-Conversation dataset mined from OpenSubtitles, covering English with German, Bengali, Indonesian, and Chinese. It evaluates several NLLB variants, ALMA-R, and instruction-tuned LLMs under zero-shot, one-shot, explanation, and contextualized prompting, reporting BLEU, CHRF++, and COMET scores. The main claims are that LLMs generally outperform NMT models on proverb translation, that conversational context improves LLM translation, that current automatic metrics are inadequate for evaluating proverb translation, and that data contamination does not bias the results. The paper also compares LLM-as-a-judge against COMET on selected model pairs.

Significance. If the central claims were established, the paper would make a useful contribution: a publicly planned proverb-translation benchmark with cultural coverage, a systematic comparison of NMT and LLM systems, and evidence about the reliability limits of standard MT metrics for figurative language. The dataset construction, including human verification and contamination analysis, is a genuine step forward. However, the headline comparison and the metric-reliability conclusion currently rest on automatic metrics whose aggregate validity is asserted rather than demonstrated, and the paper's own Section 5.1 argues that those metrics fail on proverb translation. Without human ground truth or statistical validation, the significance of the model-ranking results is not yet established.

major comments (4)
  1. [Section 4 and Section 5.1] There is an unresolved internal tension in the evidence for the central claim. The headline conclusions that 'LLMs generally outperform NMT models' and that cultural similarity helps translation are drawn exclusively from BLEU, CHRF++, and COMET scores in Tables 4 and 5 and Figure 1. Section 5.1 then argues that these same metrics are 'inadequate for reliably assessing' proverb translation. The paper never resolves this tension: it does not provide human judgments of translation quality, nor does it show that the automatic metrics are reliable enough in aggregate to rank models even when they fail on individual items. The qualitative examples in Table 6 demonstrate that the metrics can disagree with reasonable semantic equivalence, but they do not establish that the aggregate model rankings in Tables 4 and 5 are valid. The paper should supply a human-evaluation study or a metric-validity analysis (for example, correlation with human judgments on a sample) to justify the model-ranking claims.
  2. [Section 5.1] The metric-unreliability analysis is largely self-selecting. The procedure searches among 5 million hypothesis pairs for cases where the metric difference exceeds thresholds (10.0, 5.0, 10.0 for COMET, BLEU, CHRF++) while the embedding cosine difference is below 0.05. By construction this finds pairs with large metric differences and small semantic differences; it cannot estimate the prevalence or severity of metric unreliability without reporting base rates. The paper states that 22,704 pairs satisfy the thresholds, which is only about 0.45% of all pairs, yet the conclusion in the abstract is that the metrics are 'inadequate for reliably assessing' proverb translation. The analysis also assumes, without evidence, that NLLB and ALMA-R appear least often because LLMs are 'more creative' rather than because the detection thresholds are noisy. A proper treatment would report the distribution of metric-embedding disagreements, compare against a random baseline, and validate the chosen thresholds.
  3. [Section 4 and Table 4/5] All model comparisons are reported as point estimates without significance tests, confidence intervals, or error bars. Several evaluation subsets are small: for example, the PiC en-zh subset has only 272 samples, and the standalone per-direction splits in Table 3 range from roughly 250 to 400 samples. In Table 4, the difference between LLAMA-3.1 70B and GPT-4O MINI on literal from-English BLEU is 15.16 versus 13.61, and on COMET it is 72.49 versus 71.62; without paired significance testing or uncertainty quantification, these gaps may be within noise. The paper should add paired bootstrap tests, confidence intervals, or an equivalent method, and should report them for the main comparisons in Tables 4 and 5.
  4. [Section 5.2] The LLM-as-a-judge experiment does not validate either metric. The paper reports that win rates and COMET scores 'generally follow a consistent trend,' but both are automatic measurements; this consistency does not establish that either is aligned with human judgment. The conclusion that LLM-as-a-judge 'cannot fully solve the limitations of traditional evaluation methods' is reasonable, but it is not supported as stated because no human evaluation is used as the reference standard. The win-rate analysis should be interpreted as a comparison between two automatic judges, not as evidence about which judge is more accurate.
minor comments (4)
  1. [Abstract and Section 4.1] The abstract claims that 'LLMs generally outperform NMT models,' but Table 5 shows NLLB models being highly competitive on literal proverbs in PiC, and ALMA-R 13B consistently underperforms even the smallest NLLB model in several conditions; the claim should be qualified or supported by an explicit aggregate test.
  2. [Table 5] The PiC results are aggregated as 'incl. EN-DE, EN-ID, EN-ZH' and 'incl. DE-EN, ZH-EN,' which hides large per-direction differences; reporting per-direction scores alongside the aggregates would make the comparisons more interpretable.
  3. [Section 5.3 / Equation 1] The contamination metric gamma depends on the truncation proportion τ and the threshold γ>0.9, but the paper does not report the value of τ used or a sensitivity analysis; both choices should be stated and tested.
  4. [Appendix B] The annotation section reports numbers of annotators but no inter-annotator agreement or adjudication procedure; reporting agreement on a held-out sample would strengthen the dataset-quality claims.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the proverb-translation evaluation is empirically grounded, and the metric-unreliability analysis is supported by independently human-checked examples rather than by a self-referential construction.

full rationale

The paper's central claims rest on two independent empirical components. First, the model comparison in Tables 4 and 5 uses standard automatic metrics (BLEU, CHRF++, COMET) applied to a test set that was curated from MAPS (Liu et al., 2024) and OpenSubtitles with human annotation and post-editing; no model parameter is fitted to that test set, and no 'prediction' is re-derived from the fitted values. Second, the claim that automatic metrics are inadequate for proverb translation is supported by a targeted failure analysis in Section 5.1, where hypothesis pairs are selected by embedding-similarity and metric-delta thresholds and then qualitatively verified by the authors; the selected examples are shown in Table 6 and are not used as the evidence for the model rankings. The potential concern that Section 4 draws model-ranking conclusions from metrics that Section 5.1 calls unreliable is an internal evidential tension about aggregate metric validity, not a circular reduction: the rankings are not defined in terms of the inadequacy conclusion, nor vice versa. The LLM-as-a-judge experiment in Section 5.2 uses GPT-4o mini as one candidate and as judge, which is a methodological weakness, but the paper does not use win rates as a load-bearing support for the abstract's main claims, and the section's own conclusion is that LLM-as-judge 'cannot fully solve the limitations of traditional evaluation methods.' There are no load-bearing self-citations by the present authors: the dataset sources (MAPS, OpenSubtitles) and model references are external. No equation in the paper reduces a claimed output to an input by construction. Therefore, under the strict circularity standard requiring an exhibited reduction or fitted-input-renamed-as-prediction, the paper is not circular; its weaknesses are evidential and methodological rather than definitional.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The paper's conclusions rest on several hand-set thresholds and unstated reliability assumptions. The central model comparison is not derived from first principles, and no formal verification or artifact release supports reproduction. The contamination rate gamma is a new construct with no external validation.

free parameters (6)
  • Edit-distance matching threshold = 0.8
    Section 2.2.1 uses fuzzy string matching with score threshold 0.8 to find proverb-containing subtitle lines; hand-chosen.
  • Minimum quality score threshold = 4
    Section 2.2.2 sets minimum overall score of 4 for LLM-QE/DA-QE filtering; hand-chosen.
  • Maximum sample size per direction = 2000
    Section 2.2.2 caps retained samples at 2000 and derives quantile thresholds; affects data balance.
  • Metric failure detection thresholds = cosine diff < 0.05; COMET diff > 10; BLEU diff > 10; CHRF++ diff > 5
    Section 5.1 uses these thresholds to find 22,704 hypothesis pairs where metrics and semantic similarity disagree; chosen post hoc.
  • Contamination reporting threshold = gamma > 0.9
    Section 5.3 and Table 7 report percentage of samples with gamma above 0.9; hand-chosen cutoff.
  • Context window size = 5 sentences
    Section 2.2.2 retrieves up to 5 preceding and following sentences for conversational context; design choice.
assumptions (5)
  • domain assumption MAPS figurative/literal labels are accurate and transfer to proverbs used in conversation.
    Section 2.1 and Table 3 use MAPS labels for both standalone and PiC splits without re-labeling in context.
  • domain assumption Human annotations are reliable ground truth for translation quality and proverb usage.
    Section 2.2.3 and Appendix B describe annotation but report no inter-annotator agreement or adjudication.
  • domain assumption Automatic metrics are reliable enough in aggregate to rank models and languages.
    Section 4 draws all rankings from BLEU, CHRF++, and COMET without significance tests, despite Section 5.1 arguing they are unreliable.
  • ad hoc to paper The contamination rate gamma distinguishes memorization from world knowledge.
    Equation 1 in Section 5.3 defines contamination as the context-conditioned LCS gain; no external validation that this measures data contamination rather than reasoning.
  • domain assumption Embedding cosine similarity from all-mpnet-base-v2 captures semantic equivalence of proverb translations.
    Section 5.1 uses embedding similarity to identify meaning-equivalent hypotheses; no validation on proverbs.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Proverbs Run in Pairs: Evaluating Proverb Translation Capability of Large Language Model." pith.science (2026). https://pith.science/paper/E7EGDWBQ

@misc{pith2026250111953,
  author       = {Pith},
  title        = {Pith review of: Proverbs Run in Pairs: Evaluating Proverb Translation Capability of Large Language Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E7EGDWBQ}},
  note         = {Machine review of arXiv:2501.11953}
}
read the original abstract

Despite achieving remarkable performance, machine translation (MT) research remains underexplored in terms of translating cultural elements in languages, such as idioms, proverbs, and colloquial expressions. This paper investigates the capability of state-of-the-art neural machine translation (NMT) and large language models (LLMs) in translating proverbs, which are deeply rooted in cultural contexts. We construct a translation dataset of standalone proverbs and proverbs in conversation for four language pairs. Our experiments show that the studied models can achieve good translation between languages with similar cultural backgrounds, and LLMs generally outperform NMT models in proverb translation. Furthermore, we find that current automatic evaluation metrics such as BLEU, CHRF++ and COMET are inadequate for reliably assessing the quality of proverb translation, highlighting the need for more culturally aware evaluation metrics.

Figures

Figures reproduced from arXiv: 2501.11953 by the authors.

Figure 1
Figure 1. Result on proverb translation of each transla [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. We present the win rate of 5 pairs of models evaluated by the [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Proverb Translation Annotation Instruction [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Proverb Translation Annotation Interface [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: OpenSubtitle Translation Annotation Interface [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: The prompt template of subtitle translation. [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: The prompt template of LLM-based evaluation. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Easy to Complete, Hard to Choose: Investigating LLM Performance on the ProverbIT Benchmark

    cs.CL 2026-08 conditional novelty 5.0 of 10

    ProverbIT shows that large language models can complete Italian proverbs but often fail to select 'none of the above' when the exact ending is absent, revealing a gap between memorized knowledge and discriminative reasoning.

Reference graph

Works this paper leans on

30 extracted references · 9 canonical work pages · cited by 1 Pith paper

  1. [1]

    Eleftherios Avramidis, Vivien Macketanz, Ursula Strohriegel, and Hans Uszkoreit. 2019. https://doi.org/10.18653/v1/W19-5351 Linguistic evaluation of G erman- E nglish machine translation using a test suite . In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), pages 445--454, Florence, Italy. Association fo...

    work page doi:10.18653/v1/w19-5351 2019

  2. [2]

    Christos Baziotis, Prashant Mathur, and Eva Hasler. 2023. https://aclanthology.org/2023.eacl-main.267 Automatic evaluation and analysis of idioms in neural machine translation . In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3682--3700, Dubrovnik, Croatia. Association for Computational...

    work page 2023

  3. [3]

    Mathieu Constant, Gülşen Eryiğit, Johanna Monti, Lonneke van der Plas, Carlos Ramisch, Michael Rosner, and Amalia Todirascu. 2017. https://doi.org/10.1162/COLI_a_00302 Multiword Expression Processing: A Survey . Computational Linguistics, 43(4):837--892

    doi:10.1162/coli_a_00302 2017

  4. [4]

    Marta R Costa-juss \`a , James Cross, Onur C elebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672

    arXiv 2022

  5. [5]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    arXiv 2024

  6. [6]

    Marzieh Fadaee, Arianna Bisazza, and Christof Monz. 2018. https://aclanthology.org/L18-1148 Examining the tip of the iceberg: A data set for idiom translation . In Proceedings of the Eleventh International Conference on Language Resources and Evaluation ( LREC 2018) , Miyazaki, Japan. European Language Resources Association (ELRA)

    work page 2018

  7. [7]

    Pierre Isabelle, Colin Cherry, and George Foster. 2017. https://doi.org/10.18653/v1/D17-1263 A challenge set approach to evaluating machine translation . In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2486--2496, Copenhagen, Denmark. Association for Computational Linguistics

    work page doi:10.18653/v1/d17-1263 2017

  8. [8]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825

    arXiv 2023

Show all 30 references
  1. [9]

    Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. https://doi.org/10.18653/v1/2020.acl-main.560 The state and fate of linguistic diversity and inclusion in the NLP world . In Proceedings of the 58th Annual Meeting of the Association for Co...

    2020 doi

  2. [10]

    Pierre Lison and J \"o rg Tiedemann. 2016. https://aclanthology.org/L16-1147 O pen S ubtitles2016: Extracting large parallel corpora from movie and TV subtitles . In Proceedings of the Tenth International Conference on Language Resources and Evaluation ( LREC '16) , pages 923-...

    2016

  3. [11]

    Changsheng Liu and Rebecca Hwa. 2016. https://doi.org/10.18653/v1/N16-1040 Phrasal substitution of idiomatic expressions . In Proceedings of the 2016 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies , pages...

    2016 doi

  4. [12]

    Chen Liu, Fajri Koto, Timothy Baldwin, and Iryna Gurevych. 2024. https://doi.org/10.18653/v1/2024.naacl-long.112 Are multilingual LLM s culturally-diverse reasoners? an investigation into multicultural proverbs and sayings . In Proceedings of the 2024 Conference of the North A...

    2024 doi

  5. [13]

    Shushen Manakhimova, Eleftherios Avramidis, Vivien Macketanz, Ekaterina Lapshinova-Koltunski, Sergei Bagdasarov, and Sebastian M \"o ller. 2023. https://doi.org/10.18653/v1/2023.wmt-1.23 Linguistically motivated evaluation of the 2023 state-of-the-art machine translation: Can ...

    2023 doi

  6. [14]

    Peter Newmark. 2003. A textbook of translation

    2003

  7. [15]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. https://doi.org/10.3115/1073083.1073135 B leu: a method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311--3...

    2002

  8. [16]

    Maja Popovi \'c . 2017. https://doi.org/10.18653/v1/W17-4770 chr F ++: words helping character n-grams . In Proceedings of the Second Conference on Machine Translation, pages 612--618, Copenhagen, Denmark. Association for Computational Linguistics

    2017 doi

  9. [17]

    Matt Post. 2018. https://doi.org/10.18653/v1/W18-6319 A call for clarity in reporting BLEU scores . In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186--191, Brussels, Belgium. Association for Computational Linguistics

    2018 doi

  10. [18]

    Vikas Raunak, Arul Menezes, Matt Post, and Hany Hassan. 2023. https://doi.org/10.18653/v1/2023.acl-short.90 Do GPT s produce less literal translations? In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 10...

    2023 doi

  11. [19]

    Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.213 COMET : A neural framework for MT evaluation . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685--2702, ...

    2020 doi

  12. [20]

    David Stap, Eva Hasler, Bill Byrne, Christof Monz, and Ke Tran. 2024. https://aclanthology.org/2024.acl-long.336 The fine-tuning paradox: Boosting translation quality without sacrificing LLM abilities . In Proceedings of the 62nd Annual Meeting of the Association for Computati...

    2024

  13. [21]

    Zhewei Sun, Richard Zemel, and Yang Xu. 2022. https://doi.org/10.18653/v1/2022.naacl-main.383 Semantically informed slang interpretation . In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techn...

    2022 doi

  14. [22]

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, L \'e onard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram \'e , et al. 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118

    2024 arXiv

  15. [23]

    Takashi Wada, Yuji Matsumoto, Timothy Baldwin, and Jey Han Lau. 2023. https://doi.org/10.18653/v1/2023.findings-acl.290 Unsupervised paraphrasing of multiword expressions . In Findings of the Association for Computational Linguistics: ACL 2023, pages 4732--4746, Toronto, Canad...

    2023 doi

  16. [24]

    Minghao Wu, Thuy-Trang Vu, Lizhen Qu, George Foster, and Gholamreza Haffari. 2024. Adapting large language models for document-level machine translation. arXiv preprint arXiv:2401.06468

    2024 arXiv

  17. [25]

    Haoran Xu, Young Jin Kim, Amr Sharaf, and Hany Hassan Awadalla. 2024 a . https://openreview.net/forum?id=farT6XXntP A paradigm shift in machine translation: Boosting translation performance of large language models . In The Twelfth International Conference on Learning Representations

    2024

  18. [26]

    Haoran Xu, Amr Sharaf, Yunmo Chen, Weiting Tan, Lingfeng Shen, Benjamin Van Durme, Kenton Murray, and Young Jin Kim. 2024 b . https://openreview.net/forum?id=51iwkioZpn Contrastive preference optimization: Pushing the boundaries of LLM performance in machine translation . In F...

    2024

  19. [27]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671

    2024 arXiv

  20. [28]

    Andrea Zaninello and Alexandra Birch. 2020. https://aclanthology.org/2020.lrec-1.471 Multiword expression aware neural machine translation . In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 3816--3825, Marseille, France. European Language Resou...

    2020

  21. [29]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  22. [30]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.