{"id":"2f590a9c-8388-48c8-8ace-e42909d6964b","arxiv_id":"2504.18080","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Combining continued pretraining with reasoning preference optimization yields a 72B Japanese medical model that keeps 0.868 accuracy on IgakuQA with and without explanation prompting, while a model without RPO drops to 0.834.","lead":"This paper builds a Japanese medical LLM by continuing pretraining on medical texts and then applying a preference-tuning method called RPO, achieving top scores on the Japanese medical licensing exam while keeping accuracy stable when the model is asked to explain its answers. Generalists may read it because it tackles a practical trust problem in medical AI, where models often become less accurate when they have to show their reasoning.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline stability result is measured only as answer accuracy; the paper never verifies that the explanation-requesting condition actually elicits coherent, correct explanations, so the 'stable reasoning' claim may rest on prompt non-compliance rather than reasoning quality.","rationale":"After reading the paper, I agree with the reader's conditional verdict and with the identified weakest assumption. The two-stage method is described with enough detail to be plausible, the release of weights is a genuine asset, and the ablation including CPT-only and CPT+DPO is the right design for isolating RPO's effect. However, the headline contribution is framed as trustworthy reasoning, not just answer accuracy. The only outcome variable reported for IgakuQA is answer accuracy; the explanation-requesting condition is used as a stress test, but the paper never inspects the text the model produces in that condition. This matters for two concrete reasons. First, stable accuracy can be achieved by a model that ignores the instruction to explain, so it does not by itself demonstrate that reasoning generation is stable. Second, even if explanations are present, accuracy stability does not entail that the explanations are correct or aligned with the chosen answer; a model can maintain accuracy while emitting misleading rationale. Neither possibility is purely hypothetical: the behavioral difference between Preferred-MedLLM-Qwen-72B and GPT-4o—stable at 0.868 versus improving to 0.881 under explanation prompting—shows that the explanation condition can interact with models in different ways, and the paper provides no output samples or analyses to disambiguate. I therefore consider this the single most load-bearing concern. The proposed check—manual or automated inspection of a sample of outputs in both conditions—would settle it. In the meantime, the conditional verdict remains appropriate; I would not reject or accept the paper on the current evidence.","tokens_in":15471,"tokens_out":4313,"duration_ms":44289,"concrete_test":"Using the released model, run the exact Section 4.1 prompts on a random sample of 200 IgakuQA questions (e.g., 40 per exam year) in both 3-shot and 3-shot-with-explanation conditions, plus the CPT-only model as control. For each w/explanation output, record (i) whether a step-by-step explanation precedes the final answer, (ii) whether the explanation is consistent with the selected option, and (iii) a factual-coherence score judged by two medically literate annotators or by comparison with official JMLE explanations. If the RPO model's outputs frequently omit genuine reasoning or contain materially incorrect steps, the stability result is a prompt-compliance artifact and the 'stabilizing reasoning' claim fails. Also report the standard deviation across exam years and seeds.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—that RPO 'stabilizes reasoning generation'—rests entirely on the observation in Table 2 that Preferred-MedLLM-Qwen-72B scores 0.868 in both the 3-shot and 3-shot-with-explanation conditions, while CPT-only drops from 0.867 to 0.834. The load-bearing assumption is that the w/explanation condition actually forces the model to produce step-by-step reasoning and that answer accuracy under that condition measures reasoning quality. The paper never reports whether outputs in the w/explanation condition contain explanations, how long they are, or whether the explanations are coherent and factually correct. If the RPO model simply ignores the explanation request and answers directly, its accuracy can remain perfectly stable while its reasoning is no better—or worse—than the baselines. The fact that GPT-4o improves from 0.866 to 0.881 in the w/explanation condition (Table 1) makes the mechanism ambiguous: the tuned model may be suppressing explanations rather than producing better ones. Without output-level inspection, 0.868/0.868 is evidence only of stable answer accuracy, not of stable reasoning. The authors' Section 5 limitations do not address this specific gap.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a two-stage fine-tuning recipe for a Japanese medical LLM: continued pretraining (CPT) on Japanese medical texts, followed by Reasoning Preference Optimization (RPO), a DPO variant with an additional NLL term. The resulting model, Preferred-MedLLM-Qwen-72B, is evaluated on the IgakuQA benchmark under standard 3-shot prompting and under a 3-shot prompt that additionally requests explanations. The headline result is that the model achieves 0.868 accuracy in both conditions, whereas the CPT-only model drops from 0.867 to 0.834 and the base model drops from 0.802 to 0.710. The authors interpret this as evidence that RPO stabilizes reasoning generation while preserving accuracy, and they also report results on other Japanese medical QA benchmarks. The model weights are released.","tokens_in":15775,"tokens_out":3965,"duration_ms":38602,"significance":"If the central claim is supported, this is a useful empirical contribution: it shows that a preference-optimization stage can prevent the accuracy drop that many models exhibit when asked to produce explanations, and it does so in a non-English, high-stakes domain with a publicly released 72B model. The ablation design is informative, separating CPT, RPO, DPO, and the combined pipeline, and the evaluation is on an external public benchmark rather than on a bespoke metric. The main significance, however, depends on whether the stability measure actually captures reasoning quality rather than just answer accuracy under a different prompt format; the manuscript currently does not supply that evidence.","major_comments":[{"comment":"The central claim that RPO 'stabilizes reasoning generation' is supported only by answer accuracy under a prompt that requests explanations. The manuscript does not report whether the models in the '3-shot w/ explanation' condition actually produce step-by-step explanations, nor does it assess the length, coherence, prompt-compliance, or factual correctness of the generated explanations. The 0.868/0.868 result is therefore compatible with the RPO model suppressing or ignoring the explanation request, and the observation that GPT-4o improves from 0.866 to 0.881 in the explanation condition makes prompt non-compliance a concrete alternative explanation. This is load-bearing because the paper's stated contribution is trustworthy reasoning, not merely stable answer accuracy. I ask for output-level analyses (e.g., compliance rate, explanation length, expert or LLM-judged quality, and error analysis) or, failing that, for the claim to be narrowed to 'stable answer accuracy under explanation-requesting prompts.'","section":"Section 4.1-4.2, Tables 1-2"},{"comment":"The paper reports no confidence intervals, significance tests, or repeated evaluation runs. The SOTA claim over GPT-4o rests on a difference of 0.002 (0.868 vs. 0.866), which is within the range of sampling noise for a five-exam benchmark, and GPT-4o's score in the explanation condition (0.881) is actually higher than the proposed model's 0.868. Additionally, the ablation shows CPT+RPO and CPT+DPO both at 0.868 in the standard setting, so the distinguishing advantage rests entirely on the explanation-condition numbers. Please report per-year scores, bootstrap confidence intervals, or a significance test to establish that the observed differences are not artifacts of a small number of questions.","section":"Tables 1-2, Section 4.1"},{"comment":"The comparison between CPT+RPO and CPT+DPO is described only as 'a comparison using standard DPO instead of RPO,' with no details on whether the same preference pairs, the same preference hierarchy, the same number of epochs, and the same hyperparameters were used. Since the paper attributes the stabilization benefit to RPO's additional NLL term, the DPO control must be matched on all other factors; otherwise the difference could be due to preference-data construction or tuning rather than to the loss function. Please specify the exact DPO configuration and, if possible, include ablations over the RPO alpha value.","section":"Section 4.2, ablation study"}],"minor_comments":[{"comment":"Please report the size and composition of the CPT corpus and the RPO preference dataset; the current description is qualitative, which makes it difficult to assess data diversity and to reproduce the pipeline.","section":"Sections 3.1-3.2"},{"comment":"The MMMLU(med, jp) column is an average over five subsets; please provide the per-subset scores, since the paper claims the model achieves 'the highest scores on the Japanese medical related tasks of MMMLU.'","section":"Table 3"},{"comment":"For the proprietary models, please state the API version and evaluation date; this is important for fair comparison with a fixed public checkpoint like Qwen2.5-72B.","section":"Table 1, footnotes"},{"comment":"The percentage drops of 11.5% and 3.8% are relative to the base-condition scores; please state this explicitly to avoid confusion with absolute percentage-point changes.","section":"Abstract and Section 4.1"},{"comment":"RPO is introduced as 'Iterative Reasoning Preference Optimization' by Pang et al., but the paper applies it for one epoch and does not describe any iterative loop; please clarify whether this is the original iterative algorithm or a non-iterative adaptation, since that affects how the method should be cited and understood.","section":"Section 3.2"},{"comment":"Please include the exact prompt templates for both the 3-shot and 3-shot-with-explanation conditions; the single example in Table A.1 is useful but does not show how the prompt is assembled with the three examples.","section":"Appendix A"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a straightforward empirical methods paper with a useful public artifact. The main risk is not internal inconsistency but whether the central claim about reasoning stability is supported by the reported measurement; adding output-level validation or reframing the claim should be achievable within the paper's scope. I do not see concerns about the citation pattern or novelty disclosure."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The real contribution here is a specific, practically usable result: on IgakuQA, adding RPO after CPT keeps the 72B model at 0.868 when the prompt asks for an explanation, while CPT+DPO drops to 0.848 and CPT-only to 0.834. That is a clean ablation and a useful recipe for anyone building specialized medical LLMs. The paper also handles leakage properly by training on JMLE through 2017 and evaluating on 2018-2022, reports several additional Japanese medical benchmarks, and releases the model weights. Citation pattern is fine; no self-citation padding.\n\nThe soft spot is exactly where the reader and the stress-test put it. The headline claim is about 'stable reasoning,' but the only measured quantity is answer accuracy under an explanation-requesting prompt. The paper never checks whether the model actually produces explanations, whether they are coherent, or whether they are factually correct. If the RPO model simply learns to ignore the instruction to explain, it can keep its accuracy while reasoning no better - or worse - than the baselines. GPT-4o improves from 0.866 to 0.881 in the explanation condition, so the mechanism is genuinely ambiguous. The Section 5 limitations list does not mention this gap. That makes the central framing somewhat overstated, even though the empirical ablation is internally consistent.\n\nSecond-order issues are minor by comparison: the SOTA claim over GPT-4o rests on a 0.002 difference with no confidence intervals or significance tests; GPT-4o actually scores higher in the w/explanation condition. No code or training data, but the weights are a concrete artifact. None of these sink the paper.\n\nFor me this deserves serious peer review. The recipe is valuable and the core comparison between CPT+RPO and CPT+DPO is exactly the kind of controlled evidence the medical LLM field needs more of. My recommendation: accept with major revision unless the authors either soften the language from 'stable reasoning' to 'stable accuracy under explanation prompting' or add output-level analysis - explanation presence, length, coherence, factual accuracy, and instruction-following rate. With that addition, it becomes a solid paper; without it, the title overpromises.","headline":"Useful, honest ablation showing RPO beats DPO for keeping accuracy under explanation prompts in Japanese medical QA, but the 'stable reasoning' claim is not backed by any check on the explanations themselves.","tokens_in":16244,"tokens_out":1999,"would_cite":true,"duration_ms":22131,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fine-tuning Qwen2.5-72B with continued pretraining followed by reasoning preference optimization produces a Japanese medical model that holds a 0.868 accuracy whether or not it is prompted to explain its answers, while baselines fall by…","keywords":["Japanese medical LLM","reasoning preference optimization","continued pretraining","IgakuQA","explanation stability","medical question answering","preference optimization","Japanese medical licensing exam"],"falsifier":"Ask two independent clinicians to judge the medical accuracy and logical coherence of the explanations the final model produces on, say, 100 randomly sampled IgakuQA questions. If the fraction of explanations judged medically sound is clearly below the 86.8% accuracy rate, or if correct answers are regularly paired with confidently wrong rationales, then the paper's claim that RPO stabilizes reasoning is not supported and what is stabilized is only answer selection.","tokens_in":15203,"feed_emoji":"🩺","tokens_out":8520,"duration_ms":74692,"temperature":0.7,"pith_summary":"Large language models used in medicine often answer accurately when asked a direct question, but their accuracy drops when asked to show their reasoning first; that instability is a barrier to clinical trust. This paper proposes a two-stage fine-tuning recipe — continued pretraining on Japanese medical exam material followed by reasoning preference optimization (RPO) — applied to a 72-billion-parameter model. The resulting Preferred-MedLLM-Qwen-72B scores 0.868 on the IgakuQA Japanese medical licensing benchmark and, unlike the base model and a continued-pretraining-only variant, keeps exactly that score when prompted to explain its answers. The paper takes this as evidence that RPO stabilizes reasoning generation while preserving accuracy, arguing that specialized high-stakes domains need optimization for reliable explanations, not just correct answers.","feed_headline":"Medical LLM keeps 86.8% accuracy when asked to show its work","feed_subtitle":"The two-stage recipe stops the explain-prompt accuracy collapse seen in baseline and CPT-only models.","key_machinery":"The load-bearing mechanism is Reasoning Preference Optimization (RPO), a DPO variant that adds a weighted negative log-likelihood term on the chosen response to the standard DPO preference loss. Trained on preference triples built from Japanese Medical Licensing Exam questions, where a curated ground-truth explanation is preferred over a model-generated correct answer and that over a wrong answer, RPO pushes the model toward high-quality reasoning pathways while keeping the chosen response's likelihood high. It is applied with QLoRA after a continued-pretraining stage on the same Japanese medical corpus, and the paper's ablation shows that RPO, not DPO, is the component that erases the explanation-prompt accuracy drop.","core_discovery":"On the paper's own terms, the central discovery is that adding Reasoning Preference Optimization after Continued Pretraining removes the accuracy collapse that otherwise occurs when a Japanese medical model is prompted to generate step-by-step explanations. Concretely, Preferred-MedLLM-Qwen-72B scores 0.868 on IgakuQA under standard three-shot prompting and the identical 0.868 when the prompt requires an explanation, while the continued-pretraining-only model drops from 0.867 to 0.834 and the base Qwen2.5-72B drops from 0.802 to 0.710. The paper also reports that the CPT+RPO model edges out GPT-4o's 0.866 in the standard setting and, unlike a CPT+DPO configuration, shows no drop under explanation prompts. The authors read this as evidence that RPO stabilizes reasoning generation, and they frame the contribution as optimizing for reliable explanations alongside answer accuracy.","pith_inferences":["The paper measures stability only through answer accuracy under explanation prompts; a natural next test is whether the explanations themselves are factually correct and coherent, since stable answer selection with misleading rationales would not constitute trustworthy reasoning.","GPT-4o actually improves its score to 0.881 under explanation prompts, so the method's distinctive contribution is to open-weight, domain-adapted models rather than to beating all generalist proprietary models on explanation robustness.","The stabilization may come from RPO's NLL term anchoring the explanation distribution to curated ground-truth explanations; a testable hypothesis is that the effect grows with the NLL weight, which the paper sets to 10.","The recipe is language- and domain-specific but could transfer to other non-English medical settings or other high-stakes fields such as law or finance, as long as a comparable corpus and a preference hierarchy over explanations can be constructed."],"forward_implications":["A medical LLM can be built that does not lose accuracy when clinicians ask for a rationale, removing a practical barrier to clinical adoption.","RPO specifically, rather than CPT or DPO alone, is the component that stabilizes reasoning; CPT+DPO still dropped from 0.868 to 0.848 under explanation prompts.","The CPT+RPO combination generalizes zero-shot to other Japanese medical QA tasks (MedQA, MedMCQA, PubMedQA, MMMLU-med), raising the average score from 0.710 to 0.716.","The stabilization is achievable on a 72B model with quantized low-rank adaptation, suggesting the recipe can scale without full fine-tuning.","Releasing the model weights lets others reproduce and build on the stabilized reasoning behavior in the Japanese medical domain."],"supporting_citations":[{"why":"Supplies the IgakuQA benchmark and the gpt-4/gpt-3.5 baseline scores that the model must beat.","marker":"[Kasai et al., 2023]"},{"why":"Defines Reasoning Preference Optimization, the preference-based training method whose stabilization effect is the central claim.","marker":"[Pang et al., 2024]"},{"why":"Provides Direct Preference Optimization, the base that RPO extends with a weighted NLL term.","marker":"[Rafailov et al., 2023]"},{"why":"Establishes continued pretraining as the method for domain adaptation, motivating Stage 1.","marker":"[Gururangan et al., 2020]"},{"why":"Defines the Qwen2.5-72B base model that the two-stage pipeline starts from.","marker":"[Qwen et al., 2024]"},{"why":"Supports the choice of CPT over instruction tuning for injecting new domain knowledge, shaping the paper's Stage 1 design.","marker":"[Gekhman et al., 2024]"},{"why":"Provides QLoRA, the quantized low-rank adaptation that makes training the 72B model on limited GPUs feasible.","marker":"[Dettmers et al., 2023]"},{"why":"Supplies the Japanese translations of MedQA, MedMCQA, and PubMedQA used in the generalization evaluation.","marker":"[Jiang et al., 2024]"}],"fun_headline_variants":["86.8% accuracy with or without explanations: medical LLM stability","Medical LLM keeps accuracy when forced to explain its reasoning","Two-stage recipe ends explain-prompt accuracy collapse in medical AI","Japanese medical LLM holds 86.8% when asked for reasoning, beats GPT-4o","New medical LLM: accurate answers stay accurate when it explains why"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that answer accuracy under an explanation-requesting prompt is a valid proxy for reliable and stable reasoning, because the paper never directly checks whether the generated explanations are factually correct, coherent, or faithful to the reasoning that produced the answer.","fun_headline_variants_meta":{"raw":{"variants":["86.8% accuracy with or without explanations: medical LLM stability","Medical LLM keeps accuracy when forced to explain its reasoning","Two-stage recipe ends explain-prompt accuracy collapse in medical AI","Japanese medical LLM holds 86.8% when asked for reasoning, beats GPT-4o","New medical LLM: accurate answers stay accurate when it explains why"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000697,"raw_usage":{"total_tokens":3186,"prompt_tokens":1019,"completion_tokens":2167,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":635,"completion_tokens_details":{"reasoning_tokens":2071}},"tokens_in":635,"tokens_out":2167,"duration_ms":15155,"temperature":1.0,"reasoning_tokens":2071,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:24:15.679661+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Ask two independent clinicians to judge the medical accuracy and logical coherence of the explanations the final model produces on, say, 100 randomly sampled IgakuQA questions. If the fraction of explanations judged medically sound is clearly below the 86.8% accuracy rate, or if correct answers are regularly paired with confidently wrong rationales, then the paper's claim that RPO stabilizes reasoning is not supported and what is stabilized is only answer selection.","supporting_citations":[],"review_version":1}