{"id":"a8665aa1-1d00-4f35-926f-ded87e40f0e7","arxiv_id":"2504.19044","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Optimizing the Pearson correlation between hypothesis likelihood and an external quality score during fine-tuning improves LLM translation quality and turns log-likelihood into a competitive reference-free quality estimator.","lead":"This paper trains translation models to make their own likelihood scores match translation quality, by maximizing the correlation between sampled hypotheses' likelihoods and quality-estimator scores. With only about 2,000 training sentences per language, small models get large translation gains and their likelihoods become usable quality estimates.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"QE 'surpasses CometKiwi' may be distillation of the training teacher; the paper never plots CometKiwi-XXL itself as the comparison baseline.","rationale":"The translation-quality half of the paper is reasonably secure: gains are consistent across 9 directions, 3 base models, on/off-policy, multiple automatic metrics, and a small but directional human study; the cross-metric Table 2 partially addresses metric gaming. The load-bearing weakness is the QE half. The paper's Eq. 3 optimizes the Pearson correlation between likelihood and CometKiwi-XXL; therefore the calibrated likelihood is, by construction, a function of that teacher's preferences. The Section 5.2 comparison to 'CometKiwi' is the key evidence that the model 'knows' quality, but omitting the teacher itself from the plot leaves open the trivial explanation that a stronger teacher was distilled into likelihood space. This does not make the method useless—distilling a strong QE signal into likelihoods is a legitimate contribution—but it changes the claim from 'emergent quality estimation' to 'likelihood as a proxy for a specific QE model.' The paper even hedges with 'some state-of-the-art' in places, but the abstract/conclusion claim is stronger. A single table adding CometKiwi-XXL to Figure 2 with confidence intervals would settle it. I therefore keep the reader's CONDITIONAL verdict: the main method is promising, but the QE contribution should be re-scoped or re-evidenced before acceptance.","tokens_in":26275,"tokens_out":8527,"duration_ms":85036,"concrete_test":"On the WMT22 en-de and en-ru MQM segments used in Figure 2, compute three Spearman correlations against MQM: (1) calibrated model average log-likelihood, (2) CometKiwi-XXL (the Eq. 3 teacher), and (3) CometKiwi-XL / CometKiwi baselines as in the paper. Report bootstrap 95% CIs. If (1) is not significantly higher than (2), the 'surpasses CometKiwi' headline is explained by teacher distillation and should be reframed. As a robustness check, repeat the comparison on a human-labeled QE set released after the teacher's training cutoff (e.g., WMT24 MQM/DA or a fresh expert evaluation) to rule out training-set overlap between the teacher and the test set.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's most surprising claim is that calibrated log-likelihood serves as a QE metric that surpasses CometKiwi (Section 5.2, Figure 2). But the training signal q(y|x) in Eq. (3) is set to CometKiwi-XXL (Section 4.2). The QE comparison plots CometKiwi-XL and 'CometKiwi' as baselines, yet never plots the actual teacher, CometKiwi-XXL, on the same WMT22 MQM data. If the calibrated likelihood becomes a near-monotone function of CometKiwi-XXL scores—which is exactly what Eq. (3) optimizes—then its Spearman correlation with human MQM is upper-bounded by the teacher's correlation (up to estimation noise). Observing that it beats an older/smaller CometKiwi model could simply reflect that the wmt23 CometKiwi-XXL teacher is itself stronger than the wmt22 CometKiwi baseline. The conclusion's claim of doing this 'without relying on human-annotated data' is also overstated: the teacher was trained on human DA judgments. The core intuition that a well-performing model 'knows' quality would require the calibrated likelihood to exceed its own teacher, not merely an older member of the same metric family. As written, the QE result is consistent with successful distillation of the teacher into likelihood space, which is useful but not the same as emergent quality estimation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a training-time method to calibrate the log-likelihoods of machine-translation hypotheses against external quality scores. For each source sentence, several hypotheses are sampled, and the model is trained with a loss that is the negative Pearson correlation between the hypothesis log-likelihoods and quality scores (e.g., CometKiwi-XXL), optionally regularized by a supervised fine-tuning term on the highest-scoring samples. The authors apply this to LLM-based translation models (ALMA and Tower, 7B and 13B) using about 2,000 training sentences per direction, and report substantial gains over the base models, over SFT and preference-optimization baselines (CPO), across multiple metrics (CometKiwi-XL, XCOMET, COMET), in both off-policy and on-policy training, and in a human evaluation on three directions. They further claim that the calibrated log-likelihood itself becomes a strong reference-free quality estimator, surpassing CometKiwi in Spearman correlation with human MQM on en-de and en-ru, and that calibration makes beam search competitive with Best-of-100 sampling. The paper includes cross-metric training experiments (Table 2), sensitivity analyses, and ablations on sampling size.","tokens_in":26571,"tokens_out":4807,"duration_ms":48847,"significance":"If the main claims hold, the method is important: it offers a simple, training-time alternative to expensive test-time decoding strategies such as Best-of-N reranking and MBR, and it makes a conceptual connection between quality optimization and quality estimation through a shared Pearson objective. The empirical scope is broad for a paper of this type: multiple base models, multiple source languages, both on- and off-policy training, cross-metric training, checkpoint- and sensitivity analyses, and human evaluation. The paper also releases code and human evaluation data. However, the most surprising claim, that calibrated likelihood surpasses CometKiwi as a quality estimator, is weakened by the fact that the training signal is itself a CometKiwi-family model (CometKiwi-XXL), and the teacher is never plotted on the same human-annotated data. The absence of error bars or multiple-seed results and the practice of selecting learning rates and checkpoints on the validation metric also temper confidence in the exact magnitudes. These issues are addressable; the core decoding-quality result is supported by many consistent, if variance-unreported, measurements.","major_comments":[{"comment":"The claim that calibrated log-likelihood surpasses CometKiwi as a QE metric is not backed by the appropriate comparison. The training signal q(y|x) in Eq. (3) is set to CometKiwi-XXL (Section 4.2), but Figure 2 plots only \"CometKiwi\" and \"CometKiwi-XL\" as baselines against human MQM; the actual teacher, CometKiwi-XXL, is never evaluated on the same WMT22 MQM segments. Since the loss directly enforces a monotone (Pearson) relationship between log-likelihood and CometKiwi-XXL scores, the calibrated likelihood's Spearman correlation with human MQM is expected to be at most the teacher's correlation (up to estimation noise). Observing superiority over an older/smaller CometKiwi checkpoint is consistent with distilling a newer, stronger teacher into likelihood space, not with the model independently knowing translation quality. Please add the teacher's own Spearman and Kendall scores on the identical WMT22 MQM data for en-de and en-ru, and rephrase the conclusion that this is achieved \"without relying on human-annotated data\" (Section 7), since CometKiwi-XXL was trained on human DA judgments.","section":"§5.2, Figure 2, Eq. (3)"},{"comment":"The headline magnitudes lack statistical support. The paper reports single runs, with learning rates searched from 1e-5 to 1e-4 and checkpoints selected by validation XCOMET on NTREX, and then reports \"the best results for all settings\"; no standard deviations, confidence intervals, or multiple seeds are given. The reported gains, e.g., +2.8 KIWI-XL and +2.7 XCOMET over TowerInstruct-Mistral-7B (Section 5.1), are treated as conclusive, but without variance estimates the reader cannot tell whether the differences between calibration and, say, CPO are significant. Please provide at least three seeds (or bootstrap confidence intervals) for the main off-policy results, and report the exact checkpoint-selection rule and the loss weighting between L_pearson and L_sft, which is not specified anywhere in Section 3 or 4.","section":"§4.2, §5.1, Tables 1 and 3"},{"comment":"The on-policy training formulation ignores the gradient through the sampling distribution. In Eq. (3) the hypotheses y_i are drawn from tilde{p}_θ (or from p_θ in the on-policy case), but the paper states that gradients are propagated only through z_θ, µ_z, and σ_z, implicitly treating the samples as fixed. If the true gradients through the sampling procedure are non-negligible, the on-policy results in Appendix F.3 optimize an unstated surrogate rather than the stated Pearson objective. Please either justify why the score-function/rep-gradient term can be dropped (with an ablation), or include it in the derivation. This matters for the claimed generality 'under different training dynamics' but is secondary to the off-policy central claim; still, the current text is under-specified.","section":"§3, Appendix F.3"}],"minor_comments":[{"comment":"The baseline 'CometKiwi' in Figure 2 is not explicitly defined. Section 4.1 lists CometKiwi-XL and CometKiwi-XXL with model versions, but Figure 2 uses a third label; please state which checkpoint (e.g., wmt22-cometkiwi-da) this refers to, for reproducibility.","section":"Figure 2 and Section 4.1"},{"comment":"The table layout mixes TowerInstruct and TowerBase rows with '+ SFT on BoN data', '+ CPO', and '+ Calibration' subrows; the caption should clarify that the '+' rows are trained on the immediately preceding base model (TowerBase or TowerInstruct), since the current text is ambiguous for readers skimming the table.","section":"Table 1 caption and rows"},{"comment":"The 'approximately 200 times faster' claim is based on a rough 10x model-size latency ratio and 100x sampling; please give the actual measured or estimated wall-clock times, or state explicitly that this is an order-of-magnitude estimate.","section":"Section 5.1, latency comparison"}],"recommendation":"major_revision","confidential_remarks":"The paper is potentially publishable after revision. The required addition is the teacher-baseline analysis for the QE claim and variance reporting for the headline numbers; without these, the abstract overclaims. The missing loss-weight specification is a reproducibility issue that should be fixed. I would not reject on the circularity concern alone, because the translation-quality results are internally consistent and cross-metric, but the QE claim as currently worded is stronger than the evidence supports."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: this paper is a solid empirical methods contribution with a genuinely new objective, and its main translation-quality claim largely holds. The QE subclaim is where the paper overreaches, and that overreach is partly visible in the paper's own choices.\n\nThe new bit is the loss: negative Pearson correlation between sampled hypothesis log-likelihoods and an external quality score, plus an SFT regularizer. That is not the same as DPO/CPO contrastive objectives or test-time reranking, and it is simple enough to matter. The authors apply it to Tower and ALMA in 7B/13B variants, cover nine language directions, test on- and off-policy, include human evals, and release code, data, and annotations. The gains are consistent and sometimes large, e.g., +2.8 Kiwi-XL average over TowerInstruct-Mistral-7B, and the calibrated 7B/13B models beat CPO across the board. Table 2 is the right kind of control: calibrating with COMET, XCOMET, or Kiwi-XXL all improve held-out metrics, which is decent evidence against simple metric gaming.\n\nSoft spots, in proportion. First, the headline \"comparable to Tower-70B-v2 + MBR/TRR\" is generous. On the four directions where the 70B numbers are given, the calibrated 7B is lower by about 0.7–1.8 Kiwi-XL points, and the 13B is still slightly lower. Close, but not quite matching. The abstract's \"closely approximating or even surpassing\" language should be toned down.\n\nSecond, the QE claim. The training teacher is CometKiwi-XXL, yet the QE plots compare against CometKiwi-XL and the older CometKiwi, never the teacher itself. If the calibrated likelihood had a Spearman correlation with human MQM above the teacher's, that would be a real emergence story. As presented, the result is consistent with distillation of the teacher's preferences into likelihood space. The authors should plot XXL's own Spearman correlation on the same WMT22 MQM data and state directly whether their likelihood exceeds or only approaches the teacher. Relatedly, \"without relying on human-annotated data\" is misleading: the teacher was trained on human direct assessment judgments, so the claim should be \"no additional human data beyond the teacher.\"\n\nThird, methodology hygiene: no error bars or multiple seeds, learning rate and checkpoint selected on the validation set, and the SFT loss-weighting coefficient is not stated. These are fixable but need to be in the paper.\n\nOverall, the central argument about calibration improving MAP decoding and making beam search competitive with expensive sampling is well supported. The QE overreach is a substantial caveat, not a fatal flaw.\n\nThis paper deserves a serious referee. I would send it out, with a request to add the teacher baseline to the QE plots, add error bars, disclose the loss weight, and temper the claims. For an MT or LLM post-training researcher, it is a useful read.","headline":"A genuinely new training-time calibration objective with broad, consistent MT gains; the QE claim is the soft spot, since the paper likely sells teacher distillation as emergent quality estimation and never plots the actual teacher.","tokens_in":27113,"tokens_out":2636,"would_cite":true,"duration_ms":29195,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Optimizing the Pearson correlation between translation likelihood and external quality, on about 2,000 sentences per direction, substantially improves LLM translation and makes its likelihood a strong reference-free quality estimator.","keywords":["machine translation","LLM decoding","likelihood calibration","Pearson correlation","quality estimation","CometKiwi","beam search","preference optimization"],"falsifier":"The decisive experiment is to run the identical training recipe with the quality signal inverted—using the negative of the CometKiwi score as $q(y|x)$—and then evaluate the resulting translations with expert human MQM judgments. If human-evaluated quality still improves, the correlation mechanism itself carries the gains; if human quality tracks the teacher's sign, the method is a distillation of CometKiwi's preferences and will inherit its failures wherever the teacher is wrong. A complementary check: use the calibrated likelihood as a quality estimator on a direction or domain outside the teacher's training distribution, where any genuinely acquired quality awareness—rather than a memorized ranking—should still correlate with human judgment.","tokens_in":26022,"feed_emoji":"⚖️","tokens_out":29864,"duration_ms":233498,"temperature":0.7,"pith_summary":"Machine translation systems pick the highest-probability hypothesis, but a hypothesis's likelihood and its true quality are only weakly correlated, so beam search often lands on mediocre or pathological outputs. This paper claims the fix can happen at training time: sample several hypotheses per prompt, score each with a reference-free neural quality metric (CometKiwi-XXL), and directly maximize the Pearson correlation between the hypotheses' log-likelihoods and those quality scores. With about 2,000 training sentences per direction and low-rank (LoRA) fine-tuning, the method improves translation across nine target languages, beats contrastive preference optimization (CPO) on the same data, and lets a 7B model with plain beam search match the quality of a 70B model that reranks 100 sampled hypotheses, roughly 200 times faster. The calibrated log-likelihood then doubles as a reference-free quality estimator whose correlation with expert human MQM judgments rivals or surpasses CometKiwi on en→de and en→ru. The paper's larger thesis is that quality optimization and quality estimation are one problem: a translator that has learned what a good translation is should both produce it and recognize it in its own output.","feed_headline":"2,000 sentences per direction make a 7B translator match a 70B one","feed_subtitle":"Aligning candidate likelihoods with quality scores on 2K examples makes beam search match 100-sample reranking.","key_machinery":"The load-bearing object is the Pearson correlation loss. For each prompt, the model—or a fixed external sampler, in the off-policy variant—generates $k$ hypotheses by nucleus sampling, each carrying a log-likelihood $z_\\theta(y_i|x)$ and a quality score $q(y_i|x)$ from CometKiwi-XXL; the loss is the negative Pearson correlation between the two lists, $-\\frac{1}{k}\\sum_{i=1}^{k} \\frac{z_i-\\mu_z}{\\sigma_z}\\cdot\\frac{q_i-\\mu_q}{\\sigma_q}$, estimated under the nucleus-truncated sampling distribution. Because Pearson correlation is invariant to scale and shift, the objective never prescribes absolute likelihood values; it only reorders hypotheses by quality, which is exactly the granularity that decoding needs. The loss is differentiable and reduces to the dot product of two mean-centered, $\\ell^2$-normalized vectors, so it costs almost nothing to compute. A supervised fine-tuning term on the best-scoring hypotheses anchors the absolute scale, and nucleus sampling concentrates the correlation on the likely region of the decoding space, following prior evidence that correlations in that region are what decoding cares about.","core_discovery":"The paper's central claim is that likelihood-quality miscalibration—the reason the most probable hypotheses are often not the best translations—is cheaply repairable at training time. For each source sentence it draws $k$ hypotheses by nucleus sampling, computes each hypothesis's log-likelihood $z_\\theta(y_i|x)$ and its quality score $q(y_i|x)$ from CometKiwi-XXL, and minimizes the negative Pearson correlation between the two sets of values, with a supervised fine-tuning term on the highest-scoring hypotheses to anchor the absolute likelihood scale. On WMT24, TowerInstruct-Mistral-7B calibrated this way and decoded with beam search of width 5 reaches translation quality comparable to Tower-70B-v2 with 100-sample MBR/TRR—a 70B system that samples 100 candidates and reranks them with metric ensembles—while outperforming both supervised fine-tuning on best-of-n samples and CPO across CometKiwi-XL, XCOMET, and COMET, with corroborating human evaluation. The paper further claims that the calibrated model's average log-likelihood correlates with expert WMT22 MQM judgments better than CometKiwi itself on en→de and en→ru, and it reads this as evidence that a well-calibrated translation model inherently knows what a good translation is: quality optimization and quality estimation are two sides of one objective.","pith_inferences":["The result that calibrated likelihood 'surpasses CometKiwi' is best read as distillation: the model absorbs CometKiwi's ordering and re-expresses it in a cheaper, inference-free form. A targeted probe would evaluate the calibrated likelihood on a domain or direction where CometKiwi is known to be wrong and check whether the transferred ranking inherits the teacher's blind spots.","Because the objective is correlation rather than reward maximization, it should be insensitive to monotone rescaling or systematic offset in the quality signal; that property could make Pearson calibration useful in other conditional generation tasks—summarization, code generation, or grammatical error correction—where likelihood-quality miscalibration is also documented.","The SFT regularizer's role is under-explored relative to its importance: the Pearson term alone constrains only the ordering of likelihoods, so an ablation that varies or removes the SFT weight would reveal how much of the gain comes from absolute-scale anchoring rather than correlation.","In principle, the calibrated likelihood could replace the external metric inside test-time decoding, reranking sampled hypotheses with the model's own score; whether internal reranking matches external-metric reranking would directly measure how much of the teacher's knowledge transferred."],"forward_implications":["A 7B translator calibrated on about 2,000 sentences per direction and decoded with beam search of width 5 reaches the translation quality of a 70B system that samples 100 hypotheses and reranks them with metric ensembles (MBR/TRR), at roughly 200 times lower inference cost—expensive test-time optimization is replaceable by a short training step.","The calibrated model's average log-likelihood becomes a usable reference-free quality estimator, rivaling or surpassing CometKiwi on en→de and en→ru WMT22 MQM judgments; one model can translate and score its own output without a separate QE system.","Calibration gains are additive on top of supervised fine-tuning and hold across the evaluated metric families (CometKiwi-XL, CometKiwi-XXL, XCOMET, COMET) and in human evaluation, which the paper takes as evidence that the improvement is not mere metric hacking.","Quality keeps improving as more hypotheses per sentence are sampled during training, so the method's ceiling has not been reached at the tested sampling sizes.","Calibration makes efficient MAP decoding (beam search) competitive with sampling-based decoding, lowering the latency barrier for online translation deployment."],"supporting_citations":[{"why":"Establishes the miscalibration premise—low correlation between hypothesis likelihood and translation quality—that motivates the method, and justifies measuring correlation over likely hypotheses rather than the full output space.","marker":"[Ott et al., 2018]"},{"why":"CometKiwi, the reference-free neural quality metric whose scores are the fixed training signal for $q(y|x)$ in Eq. 3 and Section 4.2.","marker":"[Rei et al., 2022a]"},{"why":"The CometKiwi system used as the quality-estimation benchmark that the calibrated likelihood is claimed to rival or surpass in Section 5.2.","marker":"[Rei et al., 2022b]"},{"why":"Defines the Tower base and instruct models that the calibration method is applied to and improved upon.","marker":"[Alves et al., 2024]"},{"why":"Provides the headline comparison: Tower-70B-v2 with 100-sample MBR/TRR, which calibrated beam search is claimed to match at far lower inference cost.","marker":"[Rei et al., 2024a]"},{"why":"CPO, the preference-optimization method that serves as the main training-time baseline to beat on the same sampled data.","marker":"[Xu et al., 2024]"},{"why":"Flores-200 supplies the 2,009 English sentences used as the calibration training source for every language direction.","marker":"[Costa-Jussà et al., 2022]"},{"why":"The WMT22 QE corpus whose expert MQM annotations are the ground truth for the claim that calibrated likelihood surpasses CometKiwi.","marker":"[Zerva et al., 2022]"},{"why":"Supplies the thresholds linking XCOMET and COMET score gains to estimated human-judgment accuracy, used to argue that the reported improvements are humanly meaningful.","marker":"[Kocmi et al., 2024b]"}],"fun_headline_variants":["2K examples calibrate LLM translation: beam search matches 70B reranking","Calibrate likelihood with quality: 2K samples make beam search beat 100-sample reranking","Quality calibration on 2K examples turns beam search into a 70B-level translator","Beam search plus 2K quality-calibrated examples rivals 70B MBR","Calibrating translation likelihoods: 2K examples, beam search, 70B-level quality"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"CometKiwi-XXL, the neural metric that supplies every training-time quality score, is a faithful and unbiased proxy for human translation quality across all nine target languages, so that teaching the model to rank hypotheses by CometKiwi's scores improves translation itself rather than merely copying the teacher's preferences.","fun_headline_variants_meta":{"raw":{"variants":["2K examples calibrate LLM translation: beam search matches 70B reranking","Calibrate likelihood with quality: 2K samples make beam search beat 100-sample reranking","Quality calibration on 2K examples turns beam search into a 70B-level translator","Beam search plus 2K quality-calibrated examples rivals 70B MBR","Calibrating translation likelihoods: 2K examples, beam search, 70B-level quality"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000318,"raw_usage":{"total_tokens":1877,"prompt_tokens":1108,"completion_tokens":769,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":724,"completion_tokens_details":{"reasoning_tokens":651}},"tokens_in":724,"tokens_out":769,"duration_ms":7564,"temperature":1.0,"reasoning_tokens":651,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:03:05.263591+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"The decisive experiment is to run the identical training recipe with the quality signal inverted—using the negative of the CometKiwi score as $q(y|x)$—and then evaluate the resulting translations with expert human MQM judgments. If human-evaluated quality still improves, the correlation mechanism itself carries the gains; if human quality tracks the teacher's sign, the method is a distillation of CometKiwi's preferences and will inherit its failures wherever the teacher is wrong. A complementary check: use the calibrated likelihood as a quality estimator on a direction or domain outside the teacher's training distribution, where any genuinely acquired quality awareness—rather than a memorized ranking—should still correlate with human judgment.","supporting_citations":[],"review_version":1}