{"id":"8d427bb7-5598-4e4e-97e5-7302ade01296","arxiv_id":"2412.08901","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A single radiology report generator conditioned on a preference vector can shift generation towards one automatic metric over another, with modest measured score changes.","lead":"This paper trains one radiology-report model to change its output based on a user-chosen preference vector, such as emphasizing fluency versus clinical accuracy. The authors show that the model's scores on automatic metrics move in the chosen direction, but they do not test against real radiologist preferences.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Preference-control evidence covers only the training grid; 'optimal policy over the entire preference space' is unsupported, and the metric-reward loop leaves radiologist-preference validity untested.","rationale":"The reader correctly identifies the reward/evaluation circularity and the missing human validation as a serious weakness. My stress-test adds a more internal and, in my view, more directly load-bearing gap: the paper's flagship claim of an optimal policy over the entire preference space is never tested. All reported preference vectors lie on the discrete grid used during training, so the experiments demonstrate conditional generation for seen preference weights but do not establish the continuous generalization or Pareto optimality asserted in the conclusion. The concrete test above would settle that internal claim by comparing the single conditioned model against per-preference specialists at held-out preference vectors. If MPO tracks the specialist frontier, the main mechanism is credible; if it lags, the conclusion overstates what the model can do. The human-preference validity issue remains important but is a separate external-validation question; it would require a radiologist study and is not what my proposed computational check addresses. I therefore keep the reader's CONDITIONAL verdict: the paper is promising but should not be accepted as stating the strong conclusion until the generalization and optimality claim is demonstrated (and, ideally, human preference alignment is measured).","tokens_in":15680,"tokens_out":8674,"duration_ms":102020,"concrete_test":"On MIMIC-CXR, train K=9 specialist models, each with the same base architecture and MOO but a fixed preference vector on the grid p1=0,0.125,...,1, and also train the single MPO model. Evaluate all models at p values both on this grid and at held-out points such as p1=0.31, 0.67, and 0.83 that were not used in MPO training. Plot the (BLEU-1, F1) frontier: if the single model's points lie on or near the convex hull of the specialist models at the held-out p values, the 'entire preference space' claim is supported; otherwise the claim should be weakened to interpolation over the training grid. Report bootstrap confidence intervals over test reports.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The conclusion claims that optimizing the preference-weighted reward in Eq. (11) with randomly sampled preference vectors 'derives an optimal policy over the entire preference space.' The only direct evidence for this is Tables 1-2, which evaluate six preference vectors on a 0.2 grid, while training samples the same grid at interval 0.1 (see Implementation Details). Every tested preference vector is therefore in-distribution; no held-out preference vector, no comparison to per-preference specialist models, and no Pareto-frontier analysis is presented. Monotone movement of the training rewards along a coarse grid does not establish optimality, or even reliable interpolation, over the continuous simplex. In addition, the rewards in Eq. (9) are BLEU-1/ROUGE-L on IU-Xray and BLEU-1/F1 on MIMIC-CXR, and the same metric families are used as evaluation. Thus the preference-control results certify optimization of those automatic metrics, not the radiologist preferences named in the abstract; the supplementary RadCliQ experiment is closer, but it is still a learned metric and is not part of the main SOTA comparison.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Multi-objective Preference Optimization (MPO) for radiology report generation. MPO first trains an R2Gen-style encoder-decoder with maximum likelihood, using a Preference Vector Fusion (PVF) module to inject an m-dimensional preference vector into the visual features via attention and residual connections (Eqs. 4-5). It then applies multi-objective reinforcement learning (Eqs. 9-14) to optimize a preference-weighted sum of metric rewards: BLEU-1/ROUGE-L on IU-Xray and BLEU-1/CheXbert F1 on MIMIC-CXR, with preference vectors sampled during training. At inference, the model is conditioned on a user-specified preference vector without further fine-tuning. Experiments report preference-control tables, comparisons with prior RRG methods, ablations, a hyperparameter analysis, and supplementary three-dimensional-preference and RadCliQ experiments.","tokens_in":15917,"tokens_out":7451,"duration_ms":71937,"significance":"If the 'optimal policy over the entire preference space' claim were substantiated, the contribution would be valuable: a single RRG model whose output can be steered along multiple quality dimensions at inference time, avoiding retraining per preference. The paper's positive aspects include the ablation of PVF against simpler fusion methods, the alpha sensitivity analysis, the supplementary three-dimensional preference and RadCliQ experiments, and the efficiency comparison. However, the core evidence currently conflates optimizing automatic metrics with satisfying radiologist preferences, and all tested preference vectors lie on the training grid. With held-out preference vectors, statistical error bars, and at least a small human-preference evaluation, the contribution would be solid.","major_comments":[{"comment":"Training samples the preference vector at interval 0.1 and test uses interval 0.2, so every reported test vector (1,0), (0.8,0.2), ..., (0,1) is an in-distribution training point. The observed monotone trends therefore do not support the Conclusion's claim of an 'optimal policy over the entire preference space', which would require generalization to unobserved vectors. Please evaluate held-out preference vectors (e.g., 0.35/0.65 and 0.55/0.45), report interpolation error, and, if possible, compare against per-preference specialist models or a Pareto-frontier analysis.","section":"Effective of Preference Guidance, Tables 1-2; Implementation Details; Conclusion"},{"comment":"The RL rewards are BLEU-1 and ROUGE-L on IU-Xray and BLEU-1 and CheXbert F1 on MIMIC-CXR, and the same metric families are used as the evaluation metrics in the main and ablation tables. This makes the preference-control results a check that the model optimizes its training objectives, not that it aligns with radiologists' preferences. The supplementary RadCliQ experiment is a step in the right direction, but RadCliQ is still a learned automatic metric and is not part of the main comparison. Add a human evaluation with radiologists or clinicians rating reports generated under different preference vectors, or at least report a held-out metric not used as a reward.","section":"Eqs. (9)-(11); Tables 1-4"},{"comment":"The main comparison tables do not state the preference vector used for MPO. The MIMIC-CXR NLG row in Table 3 matches p=(1,0) of Table 2, while the CE row in Table 4 matches p=(0,1) of Table 2; on IU-Xray the Table 3 row is close to but not identical to the p=(1,0) row in Table 1 (ROUGE-L 0.415 vs. 0.413). Please disclose the exact preference vector(s) used for each reported number, and avoid mixing different vectors across tables in a way that makes the state-of-the-art claim difficult to interpret.","section":"Tables 3 and 4"},{"comment":"None of the preference-control results include error bars or significance tests. The IU-Xray ROUGE-L movement is only 0.413 to 0.415 (0.002 absolute), which is likely within seed-level noise. Report means and standard deviations over multiple random seeds or bootstrap confidence intervals before claiming that the model 'customizes' its behavior.","section":"Tables 1 and 2"},{"comment":"The per-step reward r_i^t = r_i(Y_1:t) - r_i(Y_1:t-1) is not well-defined for n-gram metrics and clinical labelers when the prefix is shorter than the n-gram order or is an incomplete sentence (e.g., BLEU-4 for t<4, CheXbert on partial text). Since the total telescopes to the sequence reward sum_i p_i r_i(Y), the stepwise decomposition is mathematically redundant; either remove it and use the sequence-level reward directly, or justify why the incremental reward is preferred.","section":"Eq. (8)"}],"minor_comments":[{"comment":"'METOR' should be 'METEOR'.","section":"Evaluation Metrics"},{"comment":"'The sampling interval of the preference vector is 0.1' conflicts with the statement in Methods and Abstract that preference vectors are 'randomly sampled' from the preference space; specify whether training uses a finite grid at resolution 0.1 or continuous i.i.d. sampling.","section":"Implementation Details"},{"comment":"The caption says 'clinical metrics and diversity scores' but no diversity score is reported; remove the phrase or add the diversity measure.","section":"Table 4 caption"},{"comment":"On IU-Xray, MPO's ROUGE-L is 0.415 in Table 3 but 0.413 in Table 1 for the same p=(1,0) configuration; reconcile the discrepancy.","section":"Tables 1 and 3"},{"comment":"The sentence 'The average value of CE is improved by about 9.4%' is ambiguous between relative improvement and absolute percentage-point improvement; state the calculation explicitly.","section":"Comparison with Previous Works"},{"comment":"Figure 1 contains a handwritten annotation ('2024.8.16肖婷-1') and garbled placeholder characters; provide a clean figure.","section":"Figure 1"},{"comment":"Several preference-alignment citations are arXiv preprints (Yu et al. 2024; Yuan et al. 2024; Shan et al. 2024); clarify how MPO differs from these methods in the RRG context.","section":"Related Works"},{"comment":"The qualitative preference analysis is based on a single example; consider adding quantitative factual-consistency scores (e.g., CheXbert recall for the highlighted findings) to support the claim that R1 is clinically more faithful.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The load-bearing parts of the paper are the preference-control claim and the state-of-the-art comparison. Current evidence is mostly in-distribution grid behavior and automatic-metric rewards; I recommend requesting held-out preference evaluation, error bars, and disclosure of the preference vectors used in the main tables. The authors should also be asked to release code to verify seed sensitivity; the IU-Xray differences are small. The 'first to align RRG with human preferences' novelty claim should be softened in light of the existing preference-alignment literature, though the RRG-specific application is new."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a genuinely new combination for radiology report generation — conditioning the generator on a preference vector via an attention fusion module and training with multi-objective RL over a weighted reward. The idea is clear and the ablations are helpful. But the paper oversells the preference-control result, and the main SOTA table has a reproducibility problem that needs fixing before I'd trust the central claims.\n\nWhat's new: earlier RL-based RRG methods optimize a fixed weighted sum of metrics; MPO makes the weights a conditional input, so a single model can tune its output toward different objectives at inference. The PVF attention fusion is simple and the ablation against concat/add/mul shows it does something. The preference-control tables do show monotone movement in the intended direction, which is real evidence of conditional control over automatic metrics.\n\nWhere it gets soft. First, the rewards are BLEU/ROUGE/F1 and the evaluation uses the same metric families, so the preference-control results certify optimization of those metrics, not radiologist preferences. The supplementary RadCliQ experiment helps a little but is still a learned metric and is not in the main comparison. Second, the 'optimal policy over the entire preference space' claim is unsupported. Training samples a 0.1 grid and evaluation uses a 0.2 grid; every tested vector is in-distribution, with no held-out preference vectors, no Pareto analysis, and no comparison to per-preference specialist models. Monotone movement on a coarse grid is not optimality or reliable interpolation.\n\nThird, and most serious: the numbers in Table 3 do not match any row in Table 1. Table 3 lists MPO with B1=0.548 and RG-L=0.415; Table 1 has B1=0.548 only at p=[1,0] where RG-L=0.413, and RG-L=0.415 only at p=[0,1] where B1=0.531. The full-model row in Table 5 similarly combines B1=0.416 with F1=0.353, which come from different preference vectors in Table 2. Unless the authors disclose the exact preference vector(s) used for each comparison, this reads as cherry-picking the best per-metric numbers. That is load-bearing for the SOTA claim.\n\nVerdict: the idea deserves a serious referee. The method is plausible, the writing is clear, and the problems are fixable with disclosure, error bars, held-out preference vectors, and a human (or at least a held-out learned metric) evaluation. But as it stands, the paper should not be accepted without those revisions.\n\nRecommendation: send to peer review, but with a referee who will check table consistency.","headline":"A new preference-conditioned radiology report generator with a real idea, but the SOTA table appears to cherry-pick across preference vectors and the 'optimal over the entire preference space' claim outruns the evidence.","tokens_in":16402,"tokens_out":3993,"would_cite":false,"duration_ms":36834,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single radiology-report model can be conditioned by a preference vector to trade off fluency against clinical accuracy, and the paper shows monotone shifts on two public chest X-ray datasets.","keywords":["radiology report generation","multi-objective reinforcement learning","preference alignment","preference vector","clinical efficacy","chest X-ray","IU-Xray","MIMIC-CXR"],"falsifier":"A reader could fix a preference vector that heavily weights CheXbert-based F1, generate reports on a held-out set, and ask radiologists to rank them against reports generated with a vector heavily weighting BLEU-1; if clinician rankings do not track the weighted clinical dimension, or if a single fixed-reward baseline matches MPO at every point of the trade-off curve, the 'optimal policy over the entire preference space' claim is falsified.","tokens_in":15490,"feed_emoji":"🩻","tokens_out":5572,"duration_ms":53758,"temperature":0.7,"pith_summary":"This paper tries to establish that radiology report generation can be aligned to heterogeneous, multidimensional radiologist preferences within one model. It proposes MPO: encode preferences as a vector on a simplex, condition the image-to-report transformer on that vector via an attention-fusion module, and train with multi-objective reinforcement learning over randomly sampled preference vectors. The claimed result is an optimal policy over the entire preference space, so at inference the same model can be pointed at any preference, such as fluency versus clinical accuracy, simply by feeding the vector. This matters because different radiologists weight report qualities differently, and prior RL-based methods optimize one fixed reward and must be retrained per preference.","feed_headline":"One model, any preference: radiology reports tuned at inference","feed_subtitle":"Weighted reward optimization lets a single checkpoint favor fluency or clinical accuracy without fine-tuning.","key_machinery":"The argument rests on two coupled mechanisms. The preference vector fusion (PVF) network expands a preference vector $p \\in \\Delta^{m-1}$ to match encoded visual features and fuses them with multi-head attention, $H_j = \\text{Softmax}(P E^\\top / \\sqrt{d}) E$, followed by a residual connection $U = E + \\alpha H$, so that the preference vector conditions the decoder. The multi-objective optimization (MOO) module defines a reward $R = \\sum_i p_i r_i(Y)$ as the linear combination of per-metric rewards, and trains by REINFORCE with a self-critical baseline $b$. Training samples preference vectors uniformly across the simplex, which the paper argues yields an optimal policy over the entire preference space.","core_discovery":"The paper claims that a single RRG model can learn a policy over the full preference simplex, so that a low-dimensional preference vector $p$ conditions generation and controls trade-offs among metrics such as BLEU, ROUGE-L, and CheXbert-based F1. The central demonstration is that moving weight between BLEU-1 and F1 on MIMIC-CXR shifts measured clinical F1 from 0.316 to 0.353 while BLEU-1 drops from 0.416 to 0.392, and similar monotone trade-offs appear on IU-Xray. The authors further claim state-of-the-art NLG scores on both datasets and best precision among compared clinical metrics.","pith_inferences":["Editorial extension: if radiologist preferences are genuinely heterogeneous, this conditional-control scheme could be deployed per reader by storing a small preference vector rather than training per-reader models.","Editorial extension: the same preference-conditioning mechanism could be applied to other image-to-text clinical tasks, such as pathology or discharge summaries, by substituting the reward dimensions.","Editorial extension: the paper implicitly assumes a linear mapping between metric weights and human value; a direct test of whether radiologists' rankings track the vector-weighted metric remains an open question.","Editorial extension: the method's 'optimal policy over the entire preference space' claim could be tested by comparing against a portfolio of fixed-reward RL models at many preference points; if the fixed-reward models dominate the conditional model, the claim would be weakened."],"forward_implications":["The same checkpoint can generate reports tuned to different preference trade-offs at inference, because the preference vector is an input condition rather than a retraining target.","Raising the preference weight on clinical measures, such as CheXbert-based F1, increases clinical precision, recall, and F1 at the cost of some NLG overlap scores; raising the NLG weight has the opposite effect.","A three-dimensional preference vector, including BLEU-1, BLEU-4, and ROUGE-L or F1 and RadCliQ, produces the best score on whichever metric receives weight one, while equal weights act as a balanced average.","The design extends to arbitrary automatic metrics as reward dimensions, since the reward is a linear weighting of per-metric improvements.","With an equal-weight preference vector, the model behaves as a balanced generator rather than one specialized to any single metric."],"supporting_citations":[{"why":"Provides the R2Gen baseline model that MPO builds on and which serves as the 'Base' in the ablation study.","marker":"Chen et al. 2020"},{"why":"CMN+RL is the closest prior RL-based RRG method optimizing a single weighted reward; MPO extends this to a preference-conditioned multi-objective policy.","marker":"Qin and Song 2022"},{"why":"Self-critical sequence training supplies the baseline reward $b$ used in Eq. (13) to stabilize the REINFORCE gradient estimator.","marker":"Rennie et al. 2017"},{"why":"REINFORCE is the policy-gradient algorithm used to optimize the preference-weighted reward, as in Eqs. (12)-(14).","marker":"Williams 1992"},{"why":"CheXbert labels clinical observations in generated and reference reports, providing the precision, recall, and F1 metrics used both as rewards and as evaluation scores.","marker":"Smit et al. 2020"},{"why":"Provides the IU-Xray dataset used for evaluation of report generation under different preference vectors.","marker":"Demner-Fushman et al. 2016"},{"why":"Provides the MIMIC-CXR dataset, used for the large-scale evaluation and for the clinical efficacy comparisons.","marker":"Johnson et al. 2019"}],"fun_headline_variants":["Preference vector steers radiology report trade-offs in one model","One model, many radiology report preferences via multi-objective RL","Adjust radiology report focus at inference with a preference dial","Single checkpoint generates reports tailored to radiologist priorities","Multi-objective preference optimization lets one model do it all"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central assumption is that radiologists' preferences can be faithfully represented as weights over the chosen automatic metrics, so that optimizing a linear combination of these rewards aligns generation with any human preference vector.","fun_headline_variants_meta":{"raw":{"variants":["Preference vector steers radiology report trade-offs in one model","One model, many radiology report preferences via multi-objective RL","Adjust radiology report focus at inference with a preference dial","Single checkpoint generates reports tailored to radiologist priorities","Multi-objective preference optimization lets one model do it all"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00023,"raw_usage":{"total_tokens":1480,"prompt_tokens":941,"completion_tokens":539,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":557,"completion_tokens_details":{"reasoning_tokens":456}},"tokens_in":557,"tokens_out":539,"duration_ms":6038,"temperature":1.0,"reasoning_tokens":456,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T17:27:20.622700+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A reader could fix a preference vector that heavily weights CheXbert-based F1, generate reports on a held-out set, and ask radiologists to rank them against reports generated with a vector heavily weighting BLEU-1; if clinician rankings do not track the weighted clinical dimension, or if a single fixed-reward baseline matches MPO at every point of the trade-off curve, the 'optimal policy over the entire preference space' claim is falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the R2Gen baseline model that MPO builds on and which serves as the 'Base' in the ablation study."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CMN+RL is the closest prior RL-based RRG method optimizing a single weighted reward; MPO extends this to a preference-conditioned multi-objective policy."},{"cited_title":"Y.; and Lungren, M","cited_arxiv_id":null,"evidence_quote":"CheXbert labels clinical observations in generated and reference reports, providing the precision, recall, and F1 metrics used both as rewards and as evaluation scores."},{"cited_title":"D.; Rosenman, M","cited_arxiv_id":null,"evidence_quote":"Provides the IU-Xray dataset used for evaluation of report generation under different preference vectors."}],"review_version":1}