Pith. sign in

REVIEW 3 major objections 4 minor 35 references

Quality-Aware Decoding: Unifying Quality Estimation and Decoding

T0 review · 3 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A token-level quality model that scores partial translations during beam search can beat re-ranking finished N-best lists with stronger post-hoc estimators, especially on long paragraphs.

desk verdict A credible new way to fuse token-level QE into beam search; the experiments hold up, but the abstract overclaims and the partial-hypothesis scoring is never directly validated. read the letter →

arxiv 2502.08561 v3 pith:4CX3YIBO submitted 2025-02-12 cs.CL

Sai Koneru , Matthias Huck , Miriam Exel , Jan Niehues This is my paper · ORCID
classification cs.CL
keywords qualityestimationneuralmachinetranslationdecodingbeamsearchtoken-levelpredictionMQMannotationsdocumentLLM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Machine translation quality is usually judged only after a translation is finished: the model produces a list of candidate translations (an N-best list) and a quality estimator picks the best candidate. If the best translation was pruned during search, no later re-ranking can recover it. This paper argues that quality estimates can instead be injected while the translation is being generated, token by token. The authors build a decoder-only quality estimation model that scores a partial hypothesis by averaging how likely each token already produced is to be error-free, and they merge that score with the translation model's own score at each beam-search step. They report that this quality-aware decoding outperforms re-ranking a finished N-best list with strong quality estimators on both tested language pairs, with the largest gains on long paragraph-level translations.

What carries the argument

The load-bearing object is a token-level uni-directional QE model: a decoder-only translation LLM (Tower) with an added classification head that outputs a GOOD/BAD probability for each token, reading only the already-generated prefix. Training labels come from WMT MQM error spans, with a masking scheme that labels only the last token of an error span BAD and masks the rest, so the model learns where an error completes rather than marking every token inside it. A partial hypothesis is scored as the average $\log P(\mathrm{GOOD}_i \mid h_{1:i}, S)$ over its tokens, which is length-normalized and lives on the same log-probability scale as the translation model's score. At each beam step, only the top $topk$ extensions per beam receive this quality score, and the merged score $\alpha \cdot \mathrm{Score}_{\mathrm{NMT}} + (1-\alpha)\cdot \mathrm{Score}_{\mathrm{QE}}$ re-ranks them; because the two models share a vocabulary, the merging stays cheap. This machinery is what lets quality feedback enter search itself rather than arriving after candidate generation.

What would settle it

A direct test: collect partial hypotheses at many prefix lengths from both good and bad final translations of the same source sentences, and compare the token-level QE score of each prefix against the human score of the completed translation. If prefixes whose continuations are poor do not receive systematically lower average GOOD probabilities than prefixes whose continuations are good, or if the ordering among prefixes flips as the hypothesis grows, then the partial-prefix signal is miscalibrated and the reported decoding gains would not generalize beyond the tested setup.

Watch

Extended reading notes

Core claim

The central claim is that a quality estimator can score a translation before it is finished—token by token, using only the words written so far—and that feeding that score into beam search yields better translations than re-ranking a list of finished candidates after the fact. The paper shows this for English→German and Chinese→English by taking a decoder-only translation model and adding a classification head that outputs a GOOD or BAD probability for each token. The quality score of a partial hypothesis is the average log-probability of its tokens being GOOD, merged with the translation model's own average log-probability through a weight $\alpha$, and this merged score re-ranks the top extensions at every beam step. The authors report that the unified decoding outperforms N-best list re-ranking with strong quality estimators across both metrics and language pairs—with gains up to 1.39 XCOMET-XXL points—and that the advantage is largest on paragraph-level inputs.

Load-bearing premise

The method assumes that judging an unfinished translation by averaging how likely each already-written word is to be error-free—using a model trained on fully annotated sentences—gives a trustworthy early signal for how good the finished translation will be.

Editorial extensions

If this is right

  • If the central claim holds, NMT systems no longer depend on the N-best list containing a good translation: quality feedback can rescue a hypothesis before an early wrong token prunes all good continuations.
  • Longer inputs stand to benefit most, because the model's own probability score and the N-best list quality degrade as the translation grows; the reported paragraph-level gain substantially exceeds the sentence-level gain.
  • A single model can serve both roles: adapting the same translation model to token-level QE keeps vocabularies aligned, so the added inference cost stays below roughly double the base decoding time.
  • Quality-aware decoding is compatible with existing post-generation methods; re-ranking or QE-fusion on top of the quality-decoded beams yields small further gains, so the approach complements rather than replaces N-best processing.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: the same partial-hypothesis GOOD probability could be used as a reward signal to fine-tune the translation model itself, shifting quality feedback from decoding time to training time; the paper mentions reinforcement learning as a possible mitigation of its added inference cost.
  • Because the masking scheme only tells the model where an error completes, the same training recipe could be applied to other span-level error annotations, and even to non-translation generation tasks where a binary per-token acceptability label is available.
  • At sentence level the reported advantage over N-best re-ranking shrinks to a small margin, so the practical value of the method is most clearly a long-document technique; short-sentence users may not find the added latency worthwhile.
  • One could test whether conditioning the QE score on document-level context, rather than only the current source sentence and prefix, widens the paragraph-level gains further; this is not evaluated in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a token-level, uni-directional quality-estimation (QE) model that can score partial translations, and integrates this model into beam search via a merged score that combines the translation model log-probability with an average GOOD-token log-probability from the QE model. The QE model is obtained by LoRA-adapting the Tower NMT model on WMT MQM data with a binary GOOD/BAD labeling scheme in which tokens inside an error span are masked and only the final token of the span is labeled BAD. Experiments on WMT23 En–De and Zh–En report that the proposed Quality-Aware Decoding improves translation quality over N-best list re-ranking with XCOMET-XL QE, with larger gains on paragraph-level inputs, and show compatibility with sampling-based strategies such as MBR and QE-Fusion.

Significance. If the central premise holds, the paper would make a genuine contribution: it is the first to show that a token-level, uni-directional QE model can be used inside decoding rather than only for post-hoc re-ranking, and it reports plausible efficiency advantages over bi-directional QE. The experimental design has notable strengths: evaluation on hold-out WMT23 test sets, significance testing on the main table via paired t-test and bootstrap resampling, a clear comparison against strong re-ranking baselines, and a thoughtful ablation on sentence versus paragraph inputs. The paper also ships code, which aids reproducibility. However, the core claim rests on an untested assumption—that the partial-hypothesis QE score is reliable on incomplete prefixes—and the reported headline improvement in the abstract is not supported by the tables. These issues need to be addressed before the contribution can be considered established.

major comments (3)
  1. [Sec. 2.2.1–2.2.2, Algorithm 1 line 5, Tables 1 and 7] The central premise that the token-level QE score is reliable for partial hypotheses is never directly tested. The training scheme masks the interior tokens of an error span and assigns BAD only to the final token of the span, so a prefix that ends before the BAD token receives no penalty at that position. During decoding, however, the same model is applied to arbitrary prefixes, and the merged score in Algorithm 1 uses the average GOOD-token log-probability over the prefix. Tables 1 and 7 report correlations with human scores only on complete WMT23 sentences. I recommend adding an experiment that evaluates the QE score on partial prefixes, for example by cutting completed hypotheses at various positions and measuring the correlation between the partial QE score and the final translation quality, or by verifying that the intermediate beam candidates selected by the merged score correlate with the final metric scores. Without such evidence, the decoding gains in Table 2 could be specific to the model's behavior on prefixes that happen to end at token boundaries in the test set, rather than a general property of quality-aware decoding.
  2. [Abstract, Table 2, Table 4] The abstract claims 'up to 1.39 XCOMET-XXL ↑', but no table in the manuscript reports an improvement of 1.39 XCOMET-XXL points. The largest XCOMET-XXL gap in Table 2 is 1.29 (89.08 vs. 87.79), and the paragraph-level impact reported in Table 4 is δ = +1.16 (88.95 vs. 87.79). Please either correct the abstract to match the reported numbers or specify exactly which comparison and condition produce the 1.39 figure.
  3. [Sec. 3, Metrics, and Table 2] There is a potential evaluation circularity that is not addressed. The token-level QE model is trained on WMT MQM annotations, and the primary evaluation metrics (XCOMET-XXL and MetricX) are also trained on WMT MQM data; the re-ranking baseline uses XCOMET-XL QE, which likewise draws on the same annotation source. Since the decoding objective and the yardstick share training signal, the observed improvements may partly reflect metric-specific behavior rather than general translation quality. This is not a formal circularity in the derivation, but it is a correctness-risk concern. A concrete test would be human evaluation or evaluation on a held-out metric not derived from the same MQM pool. The Limitations section correctly notes that human evaluation is needed, but the paper should make this risk more explicit in the main results discussion.
minor comments (4)
  1. [Abstract and Section 1 footnote 1] The abstract gives a project-page URL (ai4lt.iar.kit.edu) while footnote 1 gives a GitHub URL for the code. Please unify the code link and make the abstract reference consistent.
  2. [Appendix A.2] The sentence 'the partial model achieves much higher correlaiton that the log probabilities' contains a typo: 'correlaiton' should be 'correlation' and 'that' should likely be 'than'.
  3. [Table 5 caption] The caption says 'wheras XCOMET-XL for QE-Fusion'; the word 'wheras' should be 'whereas'.
  4. [Algorithm 1] Line 5 is ambiguous: it writes Score_QE as (1/n) Σ log P(0_1, ..., 0_n | ...), but Section 2.2.2 describes averaging the log probabilities of tokens classified as GOOD, while Section 2.2.3 says the average probability of each token being classified as GOOD. Please clarify whether the sum runs over all tokens or only over tokens predicted as GOOD, and define the notation for the GOOD label.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the core claim is an empirical decoding gain, not a derivation forced by construction or self-citation.

full rationale

The paper's central claim is empirical: integrating a token-level QE model into beam search improves translation quality over N-best list re-ranking. The QE score is defined as the average GOOD-token log-probability from a model fine-tuned on WMT MQM labels, and the merged decoding score is a weighted combination with the NMT model's log-probability (Algorithm 1). This is a design choice, not a reduction of the target result to an input by construction. The QE model is trained on full-sentence MQM annotations and applied to partial hypotheses; while this is an unvalidated assumption that raises robustness concerns, it is not circularity because the partial-prefix behavior is not assumed in the training objective. The only self-citation, Koneru et al. (2024), supplies the beam-search re-ranking mechanism; it is a published algorithm and its use does not presuppose the paper's conclusion. Hyperparameter α is tuned on validation, which is standard and does not constitute a fitted parameter being renamed as a prediction. The shared WMT MQM training signal between the QE model and the evaluation metrics (XCOMET, MetricX) is a benchmark-alignment concern, not a circular derivation. The paper explicitly acknowledges the need for human evaluation in the Limitations section, further indicating that the reported gains are empirical rather than forced. No equation or fitted value is equivalent by definition to the paper's claimed outcome.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central method relies on a score fusion hyperparameter (alpha), a candidate limit (topk), and class-imbalance weights in the QE loss; these are chosen on validation or by hand. The labeling scheme, the transfer of full-sentence QE training to partial hypotheses, the base model's prior MQM exposure, and the cross-year validity of MQM data are unproven premises. No new entities are introduced.

free parameters (4)
  • Fusion weight alpha = not reported numerically; tuned on WMT24 validation via N-best reranking (Appendix A.3)
    Controls the balance between translation-model log-probability and QE score in the merged score (Algorithm 1, line 6); chosen to optimize validation-set quality.
  • Candidate limit topk = 5
    Limits the number of extensions per beam that receive QE scoring; balances compute and exploration; not reported as systematically swept.
  • GOOD/BAD class weights in loss = 0.05/0.95 for MQM, 0.2/0.8 for distillation
    Set by hand to counter class imbalance in token-level QE training (Appendix A.1); affects the calibration of the QE score.
  • LoRA rank and learning rate = default rank; lr=1e-5
    Standard fine-tuning choices; part of the method's reproducibility but not central to the claim.
assumptions (4)
  • ad hoc to paper MQM error spans can be converted to binary GOOD/BAD token labels by masking span-internal tokens and labeling the final token BAD.
    Section 2.2.1; this labeling scheme is designed specifically for this paper and is not independently validated.
  • ad hoc to paper The average GOOD-token log-probability over a partial hypothesis is a reliable proxy for the final translation quality.
    Section 2.2.2 and Algorithm 1, line 5; the model is never evaluated on partial sequences directly.
  • domain assumption Tower has already been exposed to MQM data during instruction tuning, so quality gains are attributable to decoding rather than new data.
    Section 3, Models; supports the causal interpretation of the experiments.
  • domain assumption Held-out WMT MQM years (train up to 2022, validate 2024, test 2023) are representative and do not introduce distribution shift that biases results.
    Section 3, Datasets; standard MT practice but an assumption about cross-year consistency.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Quality-Aware Decoding: Unifying Quality Estimation and Decoding." pith.science (2026). https://pith.science/paper/4CX3YIBO

@misc{pith2026250208561,
  author       = {Pith},
  title        = {Pith review of: Quality-Aware Decoding: Unifying Quality Estimation and Decoding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4CX3YIBO}},
  note         = {Machine review of arXiv:2502.08561}
}
abstract

Quality Estimation (QE) models for Neural Machine Translation (NMT) predict the quality of the hypothesis without having access to the reference. An emerging research direction in NMT involves the use of QE models, which have demonstrated high correlations with human judgment and can enhance translations through Quality-Aware Decoding. Although several approaches have been proposed based on sampling multiple candidate translations and picking the best candidate, none have integrated these models directly into the decoding process. In this paper, we address this by proposing a novel token-level QE model capable of reliably scoring partial translations. We build a uni-directional QE model for this, as decoder models are inherently trained and efficient on partial sequences. We then present a decoding strategy that integrates the QE model for Quality-Aware decoding and demonstrate that the translation quality improves when compared to the N-best list re-ranking with state-of-the-art QE models (up to $1.39$ XCOMET-XXL $\uparrow$). Finally, we show that our approach provides significant benefits in document translation tasks, where the quality of N-best lists is typically suboptimal. Code can be found at https://ai4lt.iar.kit.edu/english/projects\_kontextmt.php

Figures

Figures reproduced from arXiv: 2502.08561 by the authors.

Figure 1
Figure 1. Example from WMT’23 English → German #ID: 10: The paragraph begins with ’Department of Homeland Security,’ which should be translated as ’Ministerium für Innere Sicherheit.’ However, the top 25 beams do not contain the correct translation and begin with an error, making N-best list re-ranking insufficient. Although the top-5 tokens at the decoding contain the correct forms ’Inn’ or ’Inner,’ the probabilities split a… view at source ↗
Figure 2
Figure 2. Token-level label annotation scheme using the MQM error tags. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Prompts used in our experiments for translation and QE model. {src_sent} and {tgt_sent} represent [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Impact of α when re-ranking with token-level Tower QE on WMT’23 Test sets [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 18 canonical work pages

  1. [1]

    Duarte M Alves, Jos \'e Pombal, Nuno M Guerreiro, Pedro H Martins, Jo \ a o Alves, Amin Farajian, Ben Peters, Ricardo Rei, Patrick Fernandes, Sweta Agrawal, et al. 2024. Tower: An open multilingual large language model for translation-related tasks. arXiv preprint arXiv:2402.17733

    arXiv 2024

  2. [2]

    Frederic Blain, Chrysoula Zerva, Ricardo Rei, Nuno M Guerreiro, Diptesh Kanojia, Jos \'e GC de Souza, Beatriz Silva, T \^a nia Vaz, Yan Jingxuan, Fatemeh Azadi, et al. 2023. Findings of the wmt 2023 shared task on quality estimation. In Proceedings of the Eighth Conference on Machine Translation, pages 629--653

    work page 2023

  3. [3]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

  4. [4]

    Aljoscha Burchardt. 2013. Multidimensional quality metrics: a flexible system for assessing translation quality. In Proceedings of Translating and the Computer 35

    work page 2013

  5. [5]

    Haikang Deng and Colin Raffel. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.721 Reward-augmented decoding: Efficient controlled text generation with a unidirectional reward model . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 11781--11791, Singapore. Association for Computational Linguistics

    doi:10.18653/v1/2023.emnlp-main.721 2023

  6. [6]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    arXiv 2024

  7. [7]

    Bryan Eikema and Wilker Aziz. 2020. Is map decoding all you need? the inadequacy of the mode in neural machine translation. In Proceedings of the 28th International Conference on Computational Linguistics, pages 4506--4520

    work page 2020

  8. [8]

    Gon c alo RA Faria, Sweta Agrawal, Ant \'o nio Farinhas, Ricardo Rei, Jos \'e GC de Souza, and Andr \'e FT Martins. 2024. Quest: Quality-aware metropolis-hastings sampling for machine translation. arXiv preprint arXiv:2406.00049

    work page Pith review arXiv 2024

Show all 35 references
  1. [9]

    Patrick Fernandes, Ant \'o nio Farinhas, Ricardo Rei, Jos \'e GC de Souza, Perez Ogayo, Graham Neubig, and Andr \'e FT Martins. 2022. Quality-aware decoding for neural machine translation. In Proceedings of the 2022 Conference of the North American Chapter of the Association f...

    2022

  2. [10]

    Mara Finkelstein, David Vilar, and Markus Freitag. 2024. Introducing the newspalm mbr and qe dataset: Llm-generated high-quality parallel data outperforms traditional web-crawled data. In Proceedings of the Ninth Conference on Machine Translation, pages 1355--1372

    2024

  3. [11]

    Markus Freitag, George Foster, David Grangier, Viresh Ratnakar, Qijun Tan, and Wolfgang Macherey. 2021. https://arxiv.org/abs/2104.14478 Experts, errors, and context: A large-scale study of human evaluation for machine translation . Preprint, arXiv:2104.14478

    2021 arXiv

  4. [12]

    Markus Freitag, David Grangier, and Isaac Caswell. 2020. Bleu might be guilty but references are not innocent. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 61--71

    2020

  5. [13]

    Markus Freitag, David Grangier, Qijun Tan, and Bowen Liang. 2022. https://doi.org/10.1162/tacl_a_00491 High quality rather than high model probability: Minimum B ayes risk decoding with neural metrics . Transactions of the Association for Computational Linguistics, 10:811--825

    2022 doi

  6. [14]

    Markus Freitag, Nitika Mathur, Daniel Deutsch, Chi-Kiu Lo, Eleftherios Avramidis, Ricardo Rei, Brian Thompson, Frederic Blain, Tom Kocmi, Jiayi Wang, et al. 2024. Are llms breaking mt metrics? results of the wmt24 metrics shared task. In Proceedings of the Ninth Conference on ...

    2024

  7. [15]

    Nuno M Guerreiro, Ricardo Rei, Daan van Stigt, Luisa Coheur, Pierre Colombo, and Andr \'e FT Martins. 2024. xcomet: Transparent machine translation evaluation through fine-grained error detection. Transactions of the Association for Computational Linguistics, 12:979--995

    2024

  8. [16]

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2021. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations

    2021

  9. [17]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825

    2023 arXiv

  10. [18]

    Juraj Juraska, Daniel Deutsch, Mara Finkelstein, and Markus Freitag. 2024. Metricx-24: The google submission to the wmt 2024 metrics shared task. arXiv preprint arXiv:2410.03983

    2024 arXiv

  11. [19]

    Tom Kocmi, Eleftherios Avramidis, Rachel Bawden, Ond r ej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, et al. 2024. Findings of the wmt24 general machine translation shared task: The llm era is here but mt is not ...

    2024

  12. [20]

    Sai Koneru, Matthias Huck, Miriam Exel, and Jan Niehues. 2024. https://doi.org/10.18653/v1/2024.wmt-1.133 Plug, play, and fuse: Zero-shot joint decoding via word-level re-ranking across diverse vocabularies . In Proceedings of the Ninth Conference on Machine Translation, pages...

    2024 doi

  13. [21]

    Wendi Li, Wei Wei, Kaihe Xu, Wenfeng Xie, Dangyang Chen, and Yu Cheng. 2024. https://doi.org/10.18653/v1/2024.findings-naacl.111 Reinforcement learning with token-level feedback for controllable text generation . In Findings of the Association for Computational Linguistics: NA...

    2024 doi

  14. [22]

    Ricardo Rei, Ana C Farinha, Chrysoula Zerva, Daan van Stigt, Craig Stewart, Pedro Ramos, Taisiya Glushkova, Andr \'e FT Martins, and Alon Lavie. 2021. Are references really needed? unbabel-ist 2021 submission for the metrics shared task. In Proceedings of the Sixth Conference ...

    2021

  15. [23]

    Ricardo Rei, Marcos Treviso, Nuno M Guerreiro, Chrysoula Zerva, Ana C Farinha, Christine Maroti, Jos \'e GC de Souza, Taisiya Glushkova, Duarte Alves, Lu \' sa Coheur, et al. 2022. Cometkiwi: Ist-unbabel 2022 submission for the quality estimation shared task. In Proceedings of...

    2022

  16. [24]

    NLLB Team et al. 2024. Scaling neural machine translation to 200 languages. Nature, 630(8018):841

    2024

  17. [25]

    Christian Tomani, David Vilar, Markus Freitag, Colin Cherry, Subhajit Naskar, Mara Finkelstein, Xavier Garcia, and Daniel Cremers. 2024. Quality-aware translation models: Efficient generation and quality estimation in a single model. In Proceedings of the 62nd Annual Meeting o...

    2024

  18. [26]

    Marcos Treviso, Nuno Guerreiro, Sweta Agrawal, Ricardo Rei, Jos \'e Pombal, T \^a nia Vaz, Helena Wu, Beatriz Silva, Daan Stigt, and Andr \'e FT Martins. 2024. xtower: A multilingual llm for explaining and correcting translation errors. In Findings of the Association for Compu...

    2024

  19. [27]

    A Vaswani. 2017. Attention is all you need. Advances in Neural Information Processing Systems

    2017

  20. [28]

    Giorgos Vernikos and Andrei Popescu-Belis. 2024. Don't rank, combine! combining machine translation hypotheses using quality estimation. arXiv preprint arXiv:2401.06688

    2024 arXiv

  21. [29]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020 doi

  22. [30]

    Haoran Xu, Amr Sharaf, Yunmo Chen, Weiting Tan, Lingfeng Shen, Benjamin Van Durme, Kenton Murray, and Young Jin Kim. 2024. Contrastive preference optimization: Pushing the boundaries of llm performance in machine translation. arXiv preprint arXiv:2401.08417

    2024 arXiv

  23. [31]

    Kevin Yang and Dan Klein. 2021. Fudge: Controlled text generation with future discriminators. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3511--3535

    2021

  24. [32]

    Chrysoula Zerva, Fr \'e d \'e ric Blain, Jos \'e GC De Souza, Diptesh Kanojia, Sourabh Deoghare, Nuno M Guerreiro, Giuseppe Attanasio, Ricardo Rei, Constantin Orasan, Matteo Negri, et al. 2024. Findings of the quality estimation shared task at wmt 2024 are llms closing the gap...

    2024

  25. [33]

    Lily H Zhang, Hamid Dadkhahi, Mara Finkelstein, Firas Trabelsi, Jiaming Luo, and Markus Freitag. 2024. Learning from others' mistakes: Finetuning machine translation models with span-level error annotations. arXiv preprint arXiv:2410.16509

    2024 arXiv

  26. [34]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  27. [35]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.