Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

CRPO: Confidence-Reward Driven Preference Optimization for Machine Translation

T0 review · 2 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proposes CRPO, a data-selection rule for preference optimization that scores translation pairs by both reward gap and the reference model's confidence, and reports that it beats reward-only selection on ten translation directions.

desk verdict A useful empirical heuristic for MT preference-pair selection with consistent but modest gains; the loss-change derivation should be read as motivation, not proof. read the letter →

arxiv 2501.13927 v1 pith:P73THSSM submitted 2025-01-23 cs.CL cs.AIcs.CV

Guofeng Cui , Pichao Wang , Yang Liu , Zemian Ke , Zhu Liu , Vimal Bhat This is my paper · ORCID
classification cs.CLcs.AIcs.CV
keywords directpreferenceoptimizationmachinetranslationdataselectionrewardmodelconfidenceCRPOefficiencyLLMalignment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CRPO claims that the right training signal for preference-based fine-tuning of translation models is not high reward alone but pairs that are both high-stakes and hard: a preferred translation should clearly outscore a dispreferred one according to a reward model, while the untrained reference model should be comparatively confident in the bad output and unsure of the good one. The paper derives two scoring rules, CR+ and CR×, from the change and the value of the DPO loss, and selects for each source sentence the single pair with the highest score. On ten translation directions around English, fine-tuning ALMA-7B and NLLB-1.3B with DPO on these pairs gives higher COMET-family and BLEURT scores than selection by reward alone, such as RSO, RS-DPO, and MBR. If the result holds, the practical payoff is data efficiency: one well-chosen preference pair per source sentence outperforms datasets built by reward thresholds or rejection sampling.

What carries the argument

The central object is the Confidence-Reward Score (CR-Score), a scalar assigned to each candidate pair $(x, y_w, y_l)$. CR+ adds a reward term and a confidence term, while CR× multiplies them. The confidence term $\log \pi_{\mathrm{ref}}(y_l|x) - \log \pi_{\mathrm{ref}}(y_w|x)$ is the reference policy's log-likelihood difference; when positive, the model currently assigns higher probability to the dispreferred translation, marking the pair as hard. The algorithm samples $K$ translations from $\pi_{\mathrm{ref}}$ for each source, sets $y_w$ to the highest-reward candidate, and keeps the candidate pair with the largest positive CR-Score, so that the selected data maximizes both quality gap and learning difficulty. The $K=50$ scaling of the reward term in CR+ bridges the different magnitudes of reward and log-likelihood.

What would settle it

Compare the actual DPO loss change of a candidate pair after a fixed number of gradient steps with the CR+ score assigned to that pair before training; if the rank correlation is near zero, the loss-change derivation is not what drives the method. Alternatively, train on the maximum-CR+ pair versus the maximum-reward-difference pair with the same preferred sentence and show whether the gain persists when confidence is not explicitly optimized.

Watch

Extended reading notes

Core claim

The central discovery is that preference data for DPO-style translation fine-tuning should be scored by combining the reward gap with the reference policy's confidence gap, not by reward alone. The paper defines CR+ = $K(R(x,y_w)-R(x,y_l)) + \log \pi_{\mathrm{ref}}(y_l|x) - \log \pi_{\mathrm{ref}}(y_w|x)$, deriving it from the change in the DPO loss under the assumption that the post-training policy assigns probability proportional to $\exp(K R(x,y))$; and CR× as the product of the same reward and confidence terms, derived from the value of the CPO-style loss. Both scores prefer pairs where the dispreferred translation is more likely under the reference policy than the preferred one, so the selected data concentrates on mistakes the model is still making. Empirically, CRPO+ and CRPO× beat RSO, RS-DPO, MBR-BW/MBR-BMW, QE fine-tuning, and the Triplet dataset on ALMA-7B and NLLB-1.3B across ten translation directions, using only one preference pair per source sentence and reporting higher average scores on COMET-22, KIWI-22, XCOMET, KIWI-XL, and BLEURT-20.

Load-bearing premise

The load-bearing premise is the paper's direct approximation that the post-training policy assigns probability to a translation in proportion to $\exp(K \cdot R(x,y))$; if real trained-policy likelihoods do not follow that reward-Boltzmann shape, CR+ is a heuristic regularizer rather than a measured loss change.

Editorial extensions

If this is right

  • Using CRPO to build the preference dataset improves average COMET-22, KIWI-22, XCOMET, and KIWI-XL scores over reward-only selection on both ALMA-7B and NLLB-1.3B.
  • A single preference pair per source sentence selected by CR-Score is enough to outperform methods that keep several pairs per source, making the fine-tuning stage more data-efficient.
  • CRPO transfers from decoder-only LLMs to encoder-decoder models, so the selection rule does not depend on a particular architecture.
  • Mixing higher-quality external candidates, such as the Triplet dataset, with reference-sampled candidates and then applying CR-Score improves results further.
  • CRPO also improves over the Triplet dataset when the fine-tuning objective is CPO rather than DPO, though switching from DPO to CPO does not by itself improve CRPO.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the CR-Score rule is not translation-specific; any task with a pointwise reward model and a reference policy could use the same hard-pair selection for DPO, including summarization and instruction following.
  • Editorial inference: because CR+ is built on the assumption $\pi_{\theta_2}(y|x) \propto \exp(K R(x,y))$, its first term is best read as a heuristic loss-change proxy; a direct test would be to compare CR+ rankings against measured DPO loss reductions after a few gradient steps.
  • Editorial inference: the confidence term could be replaced by predictive uncertainty or ensemble disagreement, offering a testable variant that removes log-likelihood scale mismatch and the $K$ hyperparameter.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes CRPO, a data-selection method for preference optimization in machine translation. For each source sentence, 64 candidate translations are sampled from the reference policy, scored by a reward model and by the reference policy's log-likelihood. CRPO keeps the highest-reward candidate as preferred and chooses a dispreferred candidate that maximizes a combination of reward difference and reference-policy confidence difference (additive in CR+, multiplicative in CR×). The selected pairs are then used to fine-tune ALMA-7B and NLLB-1.3B with DPO (plus an SFT term). The paper reports average and per-direction results over ten translation directions, comparing against RSO, RS-DPO, MBR, QE fine-tuning, and a Triplet dataset, across COMET-family metrics and BLEURT. The authors claim that CRPO outperforms these baselines in translation quality while using only one preference pair per source sentence.

Significance. If the empirical findings are reliable, CRPO would be a simple and effective data-selection heuristic for DPO-based MT fine-tuning. The study has several strengths: it covers two model architectures, ten translation directions, and a non-COMET metric (BLEURT), and it reports per-direction tables plus ablations showing that both the reward term and the confidence term contribute. The use of held-out WMT test sets and of evaluation metrics (COMET-22, KIWI-22) not used in data selection mitigates the risk that the results are purely an artifact of optimizing the selection reward model. However, the theoretical derivation of CR+ is not sound as presented, and the empirical claims lack variance estimates or significance tests; the gains over the strongest baseline are sometimes very small.

major comments (2)
  1. [3.1, Eq. (6)-(7)] The derivation of CR+ as a loss-change criterion is internally inconsistent with the DPO objective that defines the training loss. In DPO, the KL constraint in Eq. (1) implies the policy remains in a KL ball around πref, and the implicit-reward identity gives πθ(y|x) = πref(y|x) exp(r_θ(x,y)/β). If the post-training policy is approximated in this KL-consistent form, πθ2(y|x) ∝ πref(y|x) exp(K·R(x,y)), then log πθ2(yw|x) − log πθ2(yl|x) equals log πref(yw|x) − log πref(yl|x) + K(R(x,yw) − R(x,yl)). Substituting this into Eq. (4), the πref terms cancel exactly and ΔθL reduces to K(R(x,yw) − R(x,yl)), which is precisely the reward-difference criterion of RS-DPO. The confidence term enters Eq. (7) only because Eq. (6) drops the πref factor that the KL penalty and the added SFT term are designed to preserve. Moreover, with K=50, a reward gap of 0.1 yields an exp(5) ≈ 148-fold probability ratio, which is far outside any KL-constrained reachable policy, so πθ2 as defined in Eq. (6) is not a plausible intermediate policy. The central claim that CR+ selects pairs with large DPO loss change is therefore unsupported; CR+ is an ad hoc score combining reward difference with reference-policy confusion. This is load-bearing because the paper's motivation and the naming 'Confidence-Reward' rest on this derivation.
  2. [5.4, Tables 1–2] No variance estimates, confidence intervals, significance tests, or information about random seeds are reported. Several improvements over the strongest baseline RSO are extremely small: for ALMA-7B, CRPO+ exceeds RSO by +0.0021 on KIWI22 and +0.0010 on COMET22; for NLLB-1.3B, the gains are +0.0007 on KIWI22 and +0.0003 on COMET22. Since candidate sampling from πref and DPO training are stochastic, these differences may be within run-to-run noise. The paper consistently reports higher scores across most directions and both architectures, which is suggestive, but the central empirical claim of superiority would need repeated runs or paired significance testing to be statistically supported.
minor comments (5)
  1. [Appendix A, Algorithm 1] The hyperparameter ϵ in line 9 is not defined anywhere, and the condition p(ij) − p(ijmax) + ϵ > 0 is not equivalent to the stated policy of filtering out negative CR-Scores; please clarify the filtering rule and the role of ϵ.
  2. [Section 3.3] The description of how K is estimated ('selecting reward and confidence values that best approximate a balanced contribution from both terms') is too vague; a concrete calibration procedure or a sensitivity analysis over K would make the method reproducible.
  3. [Section 5.1 and Tables 1–2] The XCOMET and KIWI-XL evaluation rows use the same model families (and possibly the same checkpoints) as the reward models used for data selection; although the paper provides COMET-22 and KIWI-22 as independent metrics, the reader should be reminded of this overlap when interpreting those two rows.
  4. [Section 5.3 and Tables 1–2] RS-DPO is evaluated at two threshold settings, while other methods are given a single setting; please state how the thresholds were selected and whether a comparable hyperparameter search was performed for CRPO.
  5. [Throughout] There are minor typographical errors, e.g., 'fune-tuning' in Section 3.1 and 'distrbution' in Section 4; a proofreading pass is recommended.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CR+/CR× follow algebraically from an explicitly stated approximation, and the main empirical claims are confirmed on evaluation metrics not used for data selection.

full rationale

The derivation chain is not circular. CR+ is obtained by substituting the paper's explicitly stated approximation πθ2(y|x) ∝ exp(K·R(x,y)) (Eq. 6) into the loss-change expression (Eq. 4), which yields Eq. (7) by algebra; the confidence term is thus an openly declared modeling choice rather than a result retroactively defined by the target outcome. CR× is likewise defined directly as the negative of a constructed loss in Eqs. (8)–(11). No parameter is fitted to the reported evaluation scores: K=50 is set once for all ten translation directions and for both ALMA-7B and NLLB-1.3B to balance reward and confidence scales, and the held-out COMET-22, KIWI-22, and BLEURT-20 results, none of which participate in preference-data construction, reproduce the same ranking. The overlap of XCOMET-XL and KIWI-XL as both reward scorers and two of the four evaluation metrics is a measurement-overlap concern, but it does not make the derivation circular, and the paper explicitly addresses it in Appendix C.2 with a non-COMET metric. There is no load-bearing self-citation, imported uniqueness theorem, or fitted quantity renamed as a prediction. The main weakness—that Eq. (6) is an approximation rather than a consequence of DPO—is a correctness and robustness concern, not a circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central contribution rests on a small number of assumptions. The method inherits DPO/CPO and the Bradley-Terry setup from prior work. The main added premise is Eq. (6), where the post-training policy is replaced by a reward-Boltzmann distribution; this is an unproven modeling choice. The method also assumes reward models are reliable preference labels and that likelihood gaps identify useful hard examples. The single fitted constant K=50 and the undefined epsilon in Algorithm 1 are additional free parameters. No new physical or architectural entities are introduced.

free parameters (2)
  • K (CR+ reward weight) = 50
    Chosen to balance reward and confidence magnitudes; no validation protocol is given. See Section 3.3 and Appendix B.1.
  • epsilon in Algorithm 1 = unspecified
    Line 9 filters candidates using p(ij) - p(ijmax) + epsilon > 0; epsilon is never defined, so the exact filtering cannot be reproduced.
assumptions (4)
  • domain assumption Bradley-Terry model and DPO/CPO losses are valid for translation preference learning.
    The paper inherits this setup from Rafailov et al. (2024) and Xu et al. (2024a) in Sections 2 and 3.
  • ad hoc to paper The post-training policy can be approximated by a reward-Boltzmann distribution exp(K * R(x,y)) / Z.
    Eq. (6) states this direct approximation without derivation; it is the bridge from loss change to CR+.
  • domain assumption Reward-model scores from XCOMET-XL and wmt23-cometkiwi-da-xl reliably order translation quality.
    Preference labels are built entirely from averaged COMET reward scores in Section 5.1, with no human annotation.
  • ad hoc to paper Reference-policy log-likelihood differences identify sentence pairs with high learning potential.
    The confidence term in Eq. (7) and Eq. (11) assumes that a high-likelihood low-reward dispreferred sentence is a useful hard example.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CRPO: Confidence-Reward Driven Preference Optimization for Machine Translation." pith.science (2026). https://pith.science/paper/P73THSSM

@misc{pith2026250113927,
  author       = {Pith},
  title        = {Pith review of: CRPO: Confidence-Reward Driven Preference Optimization for Machine Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P73THSSM}},
  note         = {Machine review of arXiv:2501.13927}
}
read the original abstract

Large language models (LLMs) have shown great potential in natural language processing tasks, but their application to machine translation (MT) remains challenging due to pretraining on English-centric data and the complexity of reinforcement learning from human feedback (RLHF). Direct Preference Optimization (DPO) has emerged as a simpler and more efficient alternative, but its performance depends heavily on the quality of preference data. To address this, we propose Confidence-Reward driven Preference Optimization (CRPO), a novel method that combines reward scores with model confidence to improve data selection for fine-tuning. CRPO selects challenging sentence pairs where the model is uncertain or underperforms, leading to more effective learning. While primarily designed for LLMs, CRPO also generalizes to encoder-decoder models like NLLB, demonstrating its versatility. Empirical results show that CRPO outperforms existing methods such as RS-DPO, RSO and MBR score in both translation accuracy and data efficiency.

Figures

Figures reproduced from arXiv: 2501.13927 by the authors.

Figure 1
Figure 1. Comparison among RSO, RS-DPO and CRPO. RSO scores candidate responses with reward and applies [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Visualization of reward score and log πref for the entire candidate dataset, as well as preferred and dis-preferred sentences selected by CRPO+, CRPO×, RS-DPO, RSO and MBR-BW [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Reward score vs. log πref for entire dataset. answer this question, we compose a new candidate set by mixing Triplet Dataset with our generated candidate sentences from reference policy. We then apply CR-Score to construct preference dataset and fine-tune the policy with DPO. The results are shown in [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Distribution of reward scores and log πref [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. $M^2PO$: Multi-Perspective Multi-Pair Preference Optimization for Machine Translation

    cs.CL 2025-10 reject novelty 5.0 of 10

    M2PO combines a QE-plus-alignment reward with a dynamic curriculum and multi-pair DPO loss, and reports WMT21-22 gains for a 7B translation model, but the abstract's WMT23/24 9B parity claims are unsupported.

Reference graph

Works this paper leans on

35 extracted references · 7 canonical work pages · cited by 1 Pith paper

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  4. [4]

    Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Merouane Debbah, Etienne Goffinet, Daniel Heslow, Julien Launay, Quentin Malartic, et al. 2023. Falcon-40b: an open large language model with state-of-the-art performance

    work page 2023

  5. [5]

    Ralph Allan Bradley and Milton E Terry. 1952. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324--345

    1952

  6. [6]

    Yijie Chen, Yijin Liu, Fandong Meng, Yufeng Chen, Jinan Xu, and Jie Zhou. 2023. Improving translation faithfulness of large language models via augmenting instructions. arXiv preprint arXiv:2308.12674

    arXiv 2023

  7. [7]

    Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30

    2017

  8. [8]

    Marta R Costa-juss \`a , James Cross, Onur C elebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672

    arXiv 2022

Show all 35 references
  1. [9]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  2. [10]

    Mara Finkelstein, Subhajit Naskar, Mehdi Mirzazadeh, Apurva Shah, and Markus Freitag. 2023. Mbr and qe finetuning: Training-time distillation of the best and most expensive decoding methods. arXiv preprint arXiv:2309.10966

    2023 arXiv

  3. [11]

    Markus Freitag, Ricardo Rei, Nitika Mathur, Chi-kiu Lo, Craig Stewart, Eleftherios Avramidis, Tom Kocmi, George Foster, Alon Lavie, and Andr \'e FT Martins. 2022. Results of wmt22 metrics shared task: Stop using bleu--neural metrics are better and more robust. In Proceedings o...

    2022

  4. [12]

    Markus Freitag, Ricardo Rei, Nitika Mathur, Chi-kiu Lo, Craig Stewart, George Foster, Alon Lavie, and Ond r ej Bojar. 2021. Results of the wmt21 metrics shared task: Evaluating metrics with expert-based human evaluations on ted and news domain. In Proceedings of the Sixth Conf...

    2021

  5. [13]

    Nuno M Guerreiro, Ricardo Rei, Daan van Stigt, Luisa Coheur, Pierre Colombo, and Andr \'e FT Martins. 2023. xcomet: Transparent machine translation evaluation through fine-grained error detection. arXiv preprint arXiv:2310.10482

    2023 arXiv

  6. [14]

    Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, et al. 2023. Reinforced self-training (rest) for language modeling. arXiv preprint arXiv:2308.08998

    2023 arXiv

  7. [15]

    Saeed Khaki, JinJin Li, Lan Ma, Liu Yang, and Prathap Ramachandra. 2024. Rs-dpo: A hybrid rejection sampling and direct preference optimization method for alignment of large language models. arXiv preprint arXiv:2402.10038

    2024 arXiv

  8. [16]

    Tianqi Liu, Yao Zhao, Rishabh Joshi, Misha Khalman, Mohammad Saleh, Peter J Liu, and Jialu Liu. 2023. Statistical rejection sampling improves preference optimization. arXiv preprint arXiv:2309.06657

    2023 arXiv

  9. [17]

    Yu Meng, Mengzhou Xia, and Danqi Chen. 2024. Simpo: Simple preference optimization with a reference-free reward. arXiv preprint arXiv:2405.14734

    2024 arXiv

  10. [18]

    Radford M Neal. 2003. Slice sampling. The annals of statistics, 31(3):705--767

    2003

  11. [19]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2024. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36

    2024

  12. [20]

    Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. 2020. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, p...

    2020

  13. [21]

    Ricardo Rei, Nuno M Guerreiro, Jos \'e Pombal, Daan van Stigt, Marcos Treviso, Luisa Coheur, Jos \'e GC de Souza, and Andr \'e FT Martins. 2023. Scaling up cometkiwi: Unbabel-ist 2023 submission for the quality estimation shared task. arXiv preprint arXiv:2309.11925

    2023 arXiv

  14. [22]

    Thibault Sellam, Dipanjan Das, and Ankur P Parikh. 2020. Bleurt: Learning robust metrics for text generation. In Proceedings of ACL

    2020

  15. [23]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  16. [24]

    Qiyu Wu, Masaaki Nagata, Zhongtao Miao, and Yoshimasa Tsuruoka. 2024. Word alignment as preference for machine translation. arXiv preprint arXiv:2405.09223

    2024 arXiv

  17. [25]

    Haoran Xu, Young Jin Kim, Amr Sharaf, and Hany Hassan Awadalla. 2023. A paradigm shift in machine translation: Boosting translation performance of large language models. arXiv preprint arXiv:2309.11674

    2023 arXiv

  18. [26]

    Haoran Xu, Amr Sharaf, Yunmo Chen, Weiting Tan, Lingfeng Shen, Benjamin Van Durme, Kenton Murray, and Young Jin Kim. 2024 a . Contrastive preference optimization: Pushing the boundaries of llm performance in machine translation. arXiv preprint arXiv:2401.08417

    2024 arXiv

  19. [27]

    Nuo Xu, Jun Zhao, Can Zu, Tao Gui, Qi Zhang, and Xuanjing Huang. 2024 b . Advancing translation preference modeling with rlhf: A step towards cost-effective solution. arXiv preprint arXiv:2402.11525

    2024 arXiv

  20. [28]

    Guangyu Yang, Jinghong Chen, Weizhe Lin, and Bill Byrne. 2023 a . Direct preference optimization for neural machine translation with minimum bayes risk decoding. arXiv preprint arXiv:2311.08380

    2023 arXiv

  21. [29]

    Wen Yang, Chong Li, Jiajun Zhang, and Chengqing Zong. 2023 b . Bigtranslate: Augmenting large language models with multilingual translation capability over 100 languages. arXiv preprint arXiv:2305.18098

    2023 arXiv

  22. [30]

    Jiali Zeng, Fandong Meng, Yongjing Yin, and Jie Zhou. 2024. Teaching large language models to translate with comparison. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19488--19496

    2024

  23. [31]

    Shaolei Zhang, Qingkai Fang, Zhuocheng Zhang, Zhengrui Ma, Yan Zhou, Langlin Huang, Mengyu Bu, Shangtong Gui, Yunji Chen, Xilin Chen, et al. 2023. Bayling: Bridging cross-lingual alignment and instruction following through interactive translation for large language models. arX...

    2023 arXiv

  24. [32]

    Yao Zhao, Rishabh Joshi, Tianqi Liu, Misha Khalman, Mohammad Saleh, and Peter J Liu. 2023. Slic-hf: Sequence likelihood calibration with human feedback. arXiv preprint arXiv:2305.10425

    2023 arXiv

  25. [33]

    Yao Zhao, Mikhail Khalman, Rishabh Joshi, Shashi Narayan, Mohammad Saleh, and Peter J Liu. 2022. Calibrating sequence likelihood improves conditional language generation. In The eleventh international conference on learning representations

    2022

  26. [34]

    Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, and Lei Li. 2023 a . Multilingual machine translation with large language models: Empirical results and analysis. arXiv preprint arXiv:2304.04675

    2023 arXiv

  27. [35]

    Wenhao Zhu, Yunzhe Lv, Qingxiu Dong, Fei Yuan, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, and Lei Li. 2023 b . Extrapolating large language models to non-english by aligning languages. arXiv preprint arXiv:2308.04948

    2023 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.