{"id":"e8825bbf-577c-4504-93ff-55a2b51901af","arxiv_id":"2608.13430","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Instruction tuning raises language models' confidence and reduces the variety of their reasoning rationales, without consistent accuracy gains.","lead":"This paper compares three base language models with their instruction-tuned versions on multiple-choice question answering. It reports that instruction tuning makes models more confident and their reasoning rationales more repetitive, with little consistent gain in accuracy.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The causal attribution to 'instruction tuning' is not established because official Instruct checkpoints also differ in RLHF/DPO, data mixture, and chat-template inference.","rationale":"I agree with the reader's weakest_assumption: the official Instruct checkpoints are not cleanly matched to their base versions, so the causal attribution to instruction tuning is confounded by other post-training choices and by chat-template inference. This is load-bearing because the title, abstract, and conclusion make an explicit causal claim about the 'impact' and 'effect' of instruction tuning, not merely a descriptive claim about differences between two families of checkpoints. The descriptive results themselves appear internally consistent: confidence metrics move in the same direction across all nine model-benchmark pairs, and the controlled analysis in Table 3 shows that the 1-SelfBLEU decrease survives matching on answer choice and rationale length. Those elements give the paper real value even if the causal framing is too strong. The multiple-comparison issue and the overstated section heading about accuracy are secondary; they should be fixed but do not threaten the main descriptive findings. Because the confound is real and unresolved, the appropriate verdict remains conditional: the paper should not be accepted without either a matched SFT-only ablation or substantially softened wording that limits conclusions to 'official post-trained checkpoints differ from base checkpoints.' Since the reader already assigned CONDITIONAL for essentially this reason, my stress-test does not change the verdict.","tokens_in":11586,"tokens_out":6658,"duration_ms":74192,"concrete_test":"Train a pure-SFT instruction-tuned variant from a single base checkpoint (e.g., full fine-tune or LoRA on Llama-3.1-8B using a public instruction dataset, with no RLHF/DPO) and rerun the full Table 1 pipeline on ARC-Easy, MMLU, and CSQA with identical prompts, sampling temperature, and the same verbalized-confidence protocol. If the SFT-only variant reproduces the higher verbalized confidence and lower 1-SelfBLEU relative to the base, the causal attribution to instruction tuning is supported; if those shifts appear only after preference optimization, the paper's causal language must be revised. As an additional check, also report whether official Instruct checkpoints were evaluated with or without their chat templates, since template mismatch alone could produce the observed differences.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (title, abstract, and Section 4) is that instruction tuning itself increases confidence and lowers cross-rationale diversity. This requires the premise, introduced in 'Experimental Settings' and Table 5, that each Base/Instruct pair is matched except for instruction tuning. That premise is not met: Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, and Mistral-7B-Instruct-v0.3 are the vendors' final post-trained checkpoints, which differ from their base versions in supervised data mixture, preference optimization (RLHF/DPO), and, at inference time, the chat template. The paper itself distinguishes 'instruction tuning and preference-based post-training' in Section 2 but does not control for preference optimization. Therefore, the reported consistent increases in confidence and decreases in 1-SelfBLEU are robustly demonstrated differences between two checkpoints, but the causal attribution to instruction tuning per se is not identified; the effects could arise from RLHF/DPO, data mixture, or template-aware generation. The abstract's phrases 'induced by instruction tuning' and 'impact of instruction tuning' therefore outrun the experimental design. The length- and answer-matched analysis in Table 3 strengthens the descriptive diversity finding, but it does not resolve the training-recipe confound, so the strongest causal claim remains unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper compares three base language models (Qwen2.5-7B, Mistral-7B-v0.3, Llama-3.1-8B) with their official instruction-tuned checkpoints across three multiple-choice QA benchmarks (ARC-Easy, MMLU, CommonsenseQA). It measures answer confidence via normalized choice entropy and a two-stage verbalized-confidence protocol, and it measures rationale lexical diversity via Unique-2 and 1-SelfBLEU computed over five sampled chain-of-thought rationales per question. The central empirical claims are that instruction tuning consistently increases confidence (lower choice entropy, higher verbalized confidence) with limited accuracy change, that cross-rationale diversity (1-SelfBLEU) decreases across all nine model-benchmark pairs, that surface-level diversity (Unique-2) changes heterogeneously, and that the diversity shifts persist under a same-answer, length-matched control on CommonsenseQA.","tokens_in":11852,"tokens_out":4656,"duration_ms":48269,"significance":"If the descriptive findings are taken as comparisons between base and official instruct checkpoints, the paper provides a useful and clearly presented empirical contribution: it documents a robust decrease in cross-rationale lexical diversity and a robust increase in confidence across three model families, and it includes a thoughtful matched-answer, matched-length control. The use of standard metrics, public benchmarks, and per-example significance tests is a strength, and the authors are appropriately cautious about downstream risk in the ethical statement. However, the title, abstract, and Section 4 cast the results as causal effects of instruction tuning, and that attribution is not supported by the experimental design because the official Instruct checkpoints differ from their base versions in data mixture, preference optimization, and inference template. The paper is likely to be valuable after the causal claims are either supported by controlled training experiments or reframed as descriptive differences between shipped checkpoints.","major_comments":[{"comment":"The central causal claim that 'instruction tuning' increases confidence and decreases cross-rationale diversity is not supported by the model pairs used. Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3, and Llama-3.1-8B-Instruct are vendors' final post-trained checkpoints that also differ from their base versions in supervised fine-tuning data, preference optimization (RLHF or DPO), and likely other post-training steps; the paper itself distinguishes 'instruction tuning and preference-based post-training' in §2 but never controls for the latter. The word 'matched' in the abstract and §3 is therefore misleading. The Limitations section lists only benchmark and diversity-measure scope, omitting this training-recipe confound. To support the causal framing, the authors would need either to compare checkpoints that differ only in instruction tuning (e.g., the same base with and without the SFT stage, holding preference optimization fixed) or to explicitly reframe the contribution as documenting differences between shipped base and instruct models. The descriptive finding remains interesting, but the current causal language in the title, abstract, and §4 outruns the design.","section":"Experimental Settings (§3, Table 5)"},{"comment":"The verbalized-confidence protocol fixes the answer to the likelihood-selected candidate and then prompts the model to output a single number between 0 and 1. Instruction-tuned models are trained to comply with such format restrictions, so part of the large verbalized-confidence increase (e.g., Llama from 49.2% to 90.4% on ARC-Easy) may reflect prompt-compliance behavior rather than a genuine change in calibrated confidence. The paper does not report the distribution of elicited probabilities, nor does it test an alternative elicitation that does not condition on a fixed answer. Because the choice-entropy result independently supports the confidence-increase pattern, this issue does not invalidate the paper, but it should be addressed with at least a robustness check or a discussion of this artifact before the verbalized-confidence measure is used as a load-bearing quantity.","section":"§3, Model Confidence Evaluation (verbalized confidence)"},{"comment":"The paper does not specify whether the base and instruct models were prompted with the same raw text or whether the instruct models received their native chat template (and the base models a plain prompt). If the chat template is applied only to Instruct models, the comparison conflates template effects with training effects, which is especially relevant for verbalized confidence and for the lexical diversity of generated rationales. The generation settings in Appendix B (temperature 0.7, nucleus p=1.0, max 100 tokens) are stated as fixed, but template format is part of the generation setting. The authors should report the exact prompts, including any template wrappers, for both variants and, ideally, include a control that applies the instruct template to the base model.","section":"Experimental Settings and Appendix B (inference protocol)"}],"minor_comments":[{"comment":"The phrase 'likelihood-based calibration' in the abstract is imprecise; the paper primarily reports ECE for likelihood-based and verbalized confidence, which are calibration errors rather than a single 'likelihood-based calibration' construct.","section":"Abstract and §4"},{"comment":"The color coding in Table 1 is not accessible in grayscale; consider adding explicit arrows or boldface for the Instruct rows instead of relying on color alone. The labels in Figure 1 are very compact and hard to read at page width.","section":"Table 1 and Figure 1"},{"comment":"The sentence 'the largest increases in Unique-2 are observed for Mistral on ARC-Easy and MMLU, while the largest increase occurs on CSQA' does not name the model for the CSQA increase (it is Mistral per Table 1); this should be clarified.","section":"§4, 'Instruction Tuning Induces Heterogeneous Changes...'"},{"comment":"The verbalized-confidence prompt example does not show the exact formatting for the candidate answers (e.g., whether option labels are included as 'A. [choice A]' with the bracket placeholders literally present); please show a fully concrete example.","section":"Appendix B"},{"comment":"The reference to Huang et al. (2026) lists page numbers 1–59 for a short-paper track, which looks like an error; please verify the pagination.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The empirical core is sound as a descriptive comparison, and the controlled analysis in Table 3 is a genuine strength. The main risk is that the causal wording will mislead readers in applied fields. I would encourage the editor to require either a same-pipeline control (e.g., the base model after only the instruction-tuning SFT stage, without RLHF/DPO) or a global reframing to 'post-training differences' in the title, abstract, and conclusions. If the authors choose the reframing route, the paper is close to acceptable after minor revisions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid empirical paper that documents a consistent, robust difference between official base and instruct-tuned checkpoints: confidence goes up, cross-rationale diversity (measured by 1-SelfBLEU) goes down, and the effect survives a same-answer, length-matched control on CommonsenseQA. The contribution is real, but the title and abstract oversell the causal attribution to 'instruction tuning' when the checkpoints also differ in RLHF/DPO, data mixture, and chat-template inference.\n\nWhat I liked: The joint study of confidence and diversity is genuinely new in this combination. The controlled analysis is well designed: fixing the selected answer and matching rationale length isolates the diversity shift from answer switching and verbosity. The authors also report ECE for both likelihood-based and verbalized confidence, and they put a Limitations section in the main text that acknowledges the scope (English, multiple-choice, lexical only). The patterns are consistent across all nine model-benchmark pairs, and the p-values are two-sided paired t-tests, so the descriptive result is not a fluke.\n\nWhere it gets soft: the causal claim outruns the design. The three 'Instruct' checkpoints are the vendors' final post-trained models; they differ from base in SFT data, preference optimization, and inference template. The paper mentions preference-based post-training in Related Work but does not separate it from instruction tuning. So 'instruction tuning increases confidence' should be reframed as 'instruct-tuned checkpoints we tested show higher confidence and lower rationale diversity than their base counterparts.' That is still a useful finding, but it changes the takeaway. Also: multiple paired t-tests across nine settings are not corrected; p<0.01 helps, but a Bonferroni or FDR note would be cleaner. Minor: one sentence in Section 4 says Qwen's 'verbalized-confidence on ARC-Easy decreases from 35.3 to 22.8'—Table 7 shows that's verbalized ECE, not confidence; the wording is wrong and should be fixed. The rest of the numbers check out as far as I can tell.\n\nBottom line: worth a serious referee. The empirical core is sound and the controlled analysis is a step beyond the usual base-vs-instruct comparison. The authors should be asked to soften the causal language and add an explicit discussion of the RLHF/DPO and template confound before publication. I'd cite it as evidence of post-training effects on diversity, and it would be a good reading-group paper for anyone thinking about calibration and uncertainty.","headline":"A well-executed empirical study of confidence and rationale diversity in base vs instruct-tuned checkpoints, with a causal label that is slightly bigger than the experimental design supports.","tokens_in":12340,"tokens_out":2949,"would_cite":true,"duration_ms":29100,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Instruction tuning consistently raises model confidence while leaving accuracy nearly unchanged, and it uniformly reduces cross-rationale lexical diversity in generated answer rationales.","keywords":["instruction tuning","model confidence","verbalized confidence","lexical diversity","Self-BLEU","calibration","question answering","chain-of-thought"],"falsifier":"The central claim would be falsified by a single matched base/instruct pair in which instruction tuning lowered verbalized confidence or raised 1-SelfBLEU, or by a controlled re-run where the same instruction-tuning recipe on the same data produced no increase in verbalized confidence. Concretely, one could take one of the three base models, apply a purely supervised instruction-tuning step on identical data to the official checkpoint, and check whether choice entropy and verbalized confidence move in the same direction as reported in Table 1; if they do not, the attribution of the effect to instruction tuning as such fails.","tokens_in":11411,"feed_emoji":"🤖","tokens_out":4710,"duration_ms":42427,"temperature":0.7,"pith_summary":"The paper claims that instruction tuning, the post-training step that teaches a base language model to follow natural-language instructions, has a consistent and separable effect on how 7B-8B models express uncertainty in multiple-choice question answering. Across three model families (Qwen, Mistral, Llama) and three benchmarks (ARC-Easy, MMLU, CommonsenseQA), the tuned variants assign higher probability mass to one answer, report higher verbalized confidence, and generate rationales that look more alike from one sample to the next, while accuracy improves only slightly or not at all. The paper also shows that a surface-level diversity measure (unique bigrams) moves in different directions on different benchmarks, so the constant effect is not generic 'output becoming more uniform' but specifically a collapse in cross-rationale variability. These patterns survive when the comparison is restricted to questions where both variants choose the same answer and rationales are matched for length, which the authors take as evidence that confidence and rationale diversity are distinct signatures of instruction tuning.","feed_headline":"Instruction tuning raises confidence, lowers rationale diversity","feed_subtitle":"Tuned variants sound surer and repeat themselves more while accuracy barely moves.","key_machinery":"The argument is carried by a matched-pair comparison design and three quantitative instruments. Choice entropy $H_{\\text{choice}}(x)=-\\sum_j p_j \\log p_j/\\log M$ over the candidate answers converts likelihoods into a confidence score; a two-stage verbalized-confidence prompt extracts a numerical probability after fixing the likelihood-chosen answer; and lexical diversity is scored by Unique-2 (proportion of distinct bigrams) and 1-SelfBLEU (mean dissimilarity of each rationale to the other four sampled rationales for the same question). The controlled analysis—restricting to same-answer questions and pairing rationales by length before truncation—is what lets the paper attribute the persistence of the diversity drop to the instruction-tuned generation style rather than to different answers being chosen or to longer outputs.","core_discovery":"On its own terms, the paper's central discovery is a paired empirical contrast: moving from a base checkpoint to its instruction-tuned counterpart raises model confidence across every model–benchmark combination—choice entropy over candidate answers falls (e.g., Qwen on MMLU from 0.430 to 0.131) and verbalized confidence rises (e.g., Llama on ARC-Easy from 49.2% to 90.4%)—while accuracy changes incompletely and even stays flat (Llama on ARC-Easy at 82.2%). At the same time, cross-rationale diversity, measured as 1 minus Self-BLEU across five chain-of-thought rationales per question, falls in every setting (e.g., Mistral on ARC-Easy from 0.813 to 0.626). The instruction-tuned models are therefore not simply more fluent or narrower in one global sense: the decline is specific to how much repeated rationales resemble each other, not to how many distinct bigrams they contain, and the divergence between the two diversity measures persists under same-answer, length-matched comparisons.","pith_inferences":["The same-answer, length-matched control suggests, but does not prove, a mechanism: instruction tuning may be compressing the space of acceptable reasoning paths, which would predict that the effect is stronger on tasks with a single canonical solution than on open-ended ones—a testable extension the paper does not run.","If overconfidence and rationale homogenization share a cause in training (for example, a reward for concise, decisive outputs), then mitigation could target the training objective; the paper's paired design offers a template for such experiments.","Because the three model families differ in their post-training choices, the uniform direction of the confidence increase hints that the effect is robust to those differences, but extending the conclusion to larger or differently trained models goes beyond the 7B-8B evidence presented here."],"forward_implications":["Users of instruction-tuned QA models should expect self-reported confidence to overstate accuracy gains; a confidence jump of tens of points with flat accuracy can be a feature of post-training, not evidence of competence.","Repeated sampling from an instruction-tuned model yields rationales that are more similar to each other, so agreement across generations is a weaker signal of reliability than it would be for a base model.","Likelihood-based and verbalized confidence diverge after instruction tuning, so calibration claims based on a single confidence proxy are incomplete.","Rationale diversity and confidence are not interchangeable: controlling for the chosen answer and rationale length leaves the diversity drop intact, meaning the two capture different effects of tuning."],"supporting_citations":[{"why":"Defines the prediction as the argmax conditional likelihood and provides the question-answering calibration framing the paper adopts.","marker":"Jiang et al., 2021"},{"why":"Supplies the two-stage verbalized-confidence elicitation protocol used to obtain numerical confidence estimates.","marker":"Xiong et al., 2024"},{"why":"Provides the zero-shot chain-of-thought prompting approach ('Let's think step by step') used to generate the rationales.","marker":"Kojima et al., 2022"},{"why":"Provides the Unique Tokens Ratio and Self-BLEU metrics that define the paper's lexical-diversity measures.","marker":"Alihosseini et al., 2019"},{"why":"Introduces the ARC-Easy benchmark, one of the three evaluation sets.","marker":"Clark et al., 2018"},{"why":"Introduces the MMLU benchmark, one of the three evaluation sets.","marker":"Hendrycks et al., 2020"},{"why":"Introduces CommonsenseQA, the third evaluation set and the benchmark used for the controlled diversity analysis.","marker":"Talmor et al., 2019"},{"why":"Defines Expected Calibration Error (ECE), the calibration metric in the paper's calibration comparisons.","marker":"Guo et al., 2017"}],"fun_headline_variants":["Instruction tuning boosts confidence, thins rationale diversity","Tuned models: surer answers, more repetitive reasoning","Confidence up, rationale diversity down after tuning","Instruction tuning: overconfident and repetitive","Tuning makes models sound surer but reason alike"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that each 'Instruct' checkpoint is a faithful version of its base model with instruction tuning as the only meaningful change, so that any confidence and diversity difference can be credited to instruction tuning; in reality the tuned checkpoints also differ in data mixture and possibly in preference-based or other post-training steps.","fun_headline_variants_meta":{"raw":{"variants":["Instruction tuning boosts confidence, thins rationale diversity","Tuned models: surer answers, more repetitive reasoning","Confidence up, rationale diversity down after tuning","Instruction tuning: overconfident and repetitive","Tuning makes models sound surer but reason alike"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000486,"raw_usage":{"total_tokens":2385,"prompt_tokens":924,"completion_tokens":1461,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":540,"completion_tokens_details":{"reasoning_tokens":1398}},"tokens_in":540,"tokens_out":1461,"duration_ms":10460,"temperature":1.0,"reasoning_tokens":1398,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:10:00.129559+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"The central claim would be falsified by a single matched base/instruct pair in which instruction tuning lowered verbalized confidence or raised 1-SelfBLEU, or by a controlled re-run where the same instruction-tuning recipe on the same data produced no increase in verbalized confidence. Concretely, one could take one of the three base models, apply a purely supervised instruction-tuning step on identical data to the official checkpoint, and check whether choice entropy and verbalized confidence move in the same direction as reported in Table 1; if they do not, the attribution of the effect to instruction tuning as such fails.","supporting_citations":[],"review_version":1}