{"id":"9edc4571-fdbe-4cd2-bdff-077aa3f928c3","arxiv_id":"2510.05126","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Fine-tuning LLMs to verbalize consistency-based confidence improves calibration and discrimination on held-out and new domains, but single-task gains do not transfer between confidence-rating and pairwise comparison tasks; multitask training helps.","lead":"This paper tests whether fine-tuning large language models on their own uncertainty signals improves how honestly they report confidence, and whether that skill carries over to new topics and tasks. It finds that training helps and transfers across topics, but not between different ways of reporting confidence, unless both are trained together.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Out-of-domain calibration gains depend on the unmeasured transfer of the consistency-to-accuracy mapping a(s); TruthfulQA's ECE worsening suggests the mapping does not transfer uniformly.","rationale":"The reader's weakest assumption identifies the transfer of the consistency-to-accuracy mapping a(s) as the load-bearing condition for the cross-domain generalization claim. I agree: the method distills self-consistency into verbalized confidence using a(s) learned on training domains, so out-of-domain calibration is a joint test of whether the model can read out its consistency and whether a(s) transfers. The TruthfulQA ECE increase is an early warning that the mapping does not transfer universally, possibly due to base-rate differences induced by subsampling. This does not invalidate the strong within-domain results (e.g., GSM8K ECE 0.75 to 0.04) or the fact that MetaMedQA and LegalBench improve in several conditions, but it narrows the headline claim: fine-tuning improves calibration in out-of-domain settings only when the consistency-accuracy relationship is similar to that of the training distribution. The proposed concrete test would directly measure a_OOD(s) and consistency calibration on OOD sets, settling whether the cross-domain improvements are robust or an artifact of mapping transfer. Since the reader already assigned CONDITIONAL with the same core concern, no verdict change is needed.","tokens_in":22446,"tokens_out":5480,"duration_ms":45733,"concrete_test":"Compute consistency scores (N=10 samples) for the baseline GPT-4.1-mini on all three out-of-domain test sets. Plot empirical accuracy vs consistency bin for each OOD set (a_OOD(s)) alongside the training-domain a_train(s) used to generate fine-tuning labels (Figures A1/A3). Use a two-sample test (e.g., bootstrap) at each consistency level to check whether a_train(s) falls within the OOD confidence intervals. Additionally, compute the ECE and AUC of consistency as a direct predictor of accuracy on OOD sets; if consistency ECE on TruthfulQA is low but verbalized ECE is high, the mapping is not the culprit; if consistency ECE is also high (e.g., >0.15), the training target itself is miscalibrated for that domain, directly undermining the cross-domain generalization claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The fine-tuning target is ĉ = a(s) + ε, where a(s) is the empirical accuracy associated with self-consistency score s, estimated on the subsampled MMLU-PRO/GSM8K/TriviaQA sets (Methods: 'Generating Consistency-Based Confidence Targets'). After fine-tuning, the model's verbalized confidence is, in effect, a learned readout of its own self-consistency under the mapping a(s). Therefore, out-of-domain calibration performance tests two things jointly: (i) whether verbalized confidence tracks self-consistency, and (ii) whether the training-domain mapping a(s) also describes the consistency–accuracy relationship in TruthfulQA, MetaMedQA, and LegalBench. The paper never measures a(s) on the out-of-domain datasets, so the cross-domain claim rests on the unstated assumption that this mapping transfers. The data already hint at trouble: TruthfulQA ECE worsens from 0.10 to 0.18 after M+G+T (S) fine-tuning (Table 1, row 7), while MetaMedQA and LegalBench improve. A likely mechanism is base-rate shift: subsampling to balance consistency bins reduced training-domain accuracy to 0.31/0.23/0.56 (Table A2), and TruthfulQA baseline accuracy is 0.81 (Table A3). The model learns a lower confidence prior, producing underconfidence on high-accuracy domains. Unless the consistency–accuracy mapping is shown to transfer, the headline claim that fine-tuning 'lowers calibration error' in out-of-domain evaluations is not established; it holds only for domains whose a(s) resembles the training domains.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper tests whether supervised fine-tuning can improve LLMs' ability to communicate uncertainty, measured by single-question numeric confidence and pairwise confidence comparison. The authors fine-tune GPT-4.1-mini and Llama3.1-70B on MMLU-PRO, GSM8K, and TriviaQA using self-consistency-derived targets, then evaluate calibration (ECE) and discrimination (AUC, AUCc, AUCa) on held-out within-domain questions and out-of-domain TruthfulQA, MetaMedQA, and LegalBench. The central claims are that fine-tuning improves calibration and discrimination within and across domains; that single-task training does not transfer between the two metacognitive task formats; and that multitask training (C+S) yields broader out-of-domain gains.","tokens_in":22838,"tokens_out":6323,"duration_ms":54385,"significance":"If the results hold, the paper makes a useful empirical contribution: it shows that explicit uncertainty communication is trainable without sacrificing answer accuracy, under a single-sample inference cost, across two model families, with held-out questions and public code/data. The within-domain effects are large and consistent, and the stability of accuracy in the main conditions strengthens the AUC interpretation. However, the cross-domain calibration claim is not uniform across datasets or models, and the pairwise AUCc metric is partly self-referential because it is evaluated against the same consistency signal used to create the training labels. These issues must be addressed before the broader generalization claims are established.","major_comments":[{"comment":"","section":"Methods: Generating Consistency-Based Confidence Targets; Results: Single-Question Confidence, Table 1"},{"comment":"","section":"Methods: Confidence comparison fine-tuning; Eq. (3); Table 2"},{"comment":"","section":"Results: Single-Question Confidence; Table 1 rows 7–9; Appendix Table A6"}],"minor_comments":[{"comment":"","section":"Methods: Data Sets"},{"comment":"","section":"Appendix Tables A2 and A5"},{"comment":"","section":"Results: Multitask fine-tuning"},{"comment":"","section":"Data Availability"},{"comment":"","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The within-domain results are credible and useful, but the paper's abstract and Discussion overstate the cross-domain calibration and cross-task generalization findings. The main revisions should focus on re-analyzing or qualifying the out-of-domain claims and on separating AUCc from the self-referential training target. The data/code release is a strength."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know upfront. The core empirical results hold: consistency-based fine-tuning improves verbalized confidence — calibration and discrimination — on held-out within-domain questions, in both GPT-4.1-mini and Llama-3.1-70B. But the abstract's claim that fine-tuning 'lowers calibration error… in out-of-domain evaluations' is overbroad: TruthfulQA ECE gets worse after fine-tuning in both models (0.10→0.18; 0.08→0.18), and the paper never acknowledges that squarely.\n\nWhat is new is the 2x2 design: two metacognitive tasks (single-question confidence, pairwise comparison) crossed with domain generalization (held-out vs. medical, legal, truthfulness). Prior work trained single tasks in single domains. The finding that single-task training does not transfer across metacognitive formats, while joint C+S training partially bridges the gap, is new and fairly documented — even though the multitask benefit does not appear in Llama's pairwise AUCc, which the paper honestly reports.\n\nThe paper does several things right. It reports accuracy before and after fine-tuning, and accuracy is stable, so the AUC gains are not a difficulty confound — a strength this literature doesn't always show. Within-domain ECE drops are large (GSM8K from 0.75 to 0.04). Two model families, held-out questions, paired bootstrap CIs, and OSF data/code. That is reproducible work and should be treated as such.\n\nThe soft spots, weighted. First, the out-of-domain calibration claim. Training targets are ĉ = a(s) + ε, where a(s) is the empirical accuracy-by-consistency mapping estimated on subsampled training domains whose accuracy was deliberately lowered (0.31/0.23/0.56 on the test splits). Out-of-domain ECE therefore tests two things jointly: whether the model tracks its own consistency, and whether the training-domain a(s) mapping transfers to TruthfulQA, MetaMedQA, and LegalBench. The paper never measures a(s) on those domains. TruthfulQA's ECE worsening is exactly what you'd expect if the model learned a lower confidence prior and hit a domain with 0.81 baseline accuracy. The stress-test note lands; the cross-domain calibration claim is not established. The cross-domain discrimination (AUC) gains use independent correctness labels and do hold, so this weakens the paper without refuting it.\n\nSecond, the paper skips the obvious baseline: using the consistency score (or a(s)) directly as the confidence estimate at inference. Table A2 shows the consistency signal already has AUC 0.71–0.86 on training-domain test sets. Without that comparison, you can't tell whether fine-tuning beats the free internal signal — a real practical question given the paper's own motivation about avoiding repeated sampling.\n\nMinor: AUCc is partly self-referential since it is evaluated against the same consistency scores used to build labels, pairwise AUCa gains are inconsistent across domains, and there is no multiple-comparison correction.\n\nWho should read it: people working on LLM calibration, verbalized confidence, and deployment of uncertainty-aware assistants. It deserves a serious referee. I would send it to review and ask for the consistency-score baseline, a direct a(s) transfer check on the out-of-domain sets, and a rewording of the cross-domain calibration claim.","headline":"Solid empirical study: consistency-based fine-tuning improves verbalized confidence and the within-domain and cross-domain discrimination results hold, but the cross-domain calibration claim is overbroad and the missing consistency-score baseline is the main practical gap.","tokens_in":23312,"tokens_out":7221,"would_cite":true,"duration_ms":55952,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that supervised fine-tuning on self-consistency-based targets can align LLMs' verbalized confidence with actual accuracy, that these calibration gains transfer to unseen domains, but that single-task training does not tran","keywords":["uncertainty communication","metacognition","confidence calibration","self-consistency","supervised fine-tuning","large language models","domain generalization","discrimination"],"falsifier":"A direct check: on an unseen benchmark, compute the empirical accuracy for each consistency bin and compare it with the mapping used to create training labels. If the bin-wise accuracies differ, a model fine-tuned with the transferred mapping should show no ECE improvement (or a reversal) relative to the baseline; the paper's TruthfulQA result—where ECE rose from 0.10 to 0.18 after fine-tuning—is a partial instance of that failure.","tokens_in":22331,"feed_emoji":"🎯","tokens_out":6544,"duration_ms":49738,"temperature":0.7,"pith_summary":"The paper shows that a language model's stated confidence can be re-aligned with its true accuracy through supervised fine-tuning, using the model's own sampling consistency as the training signal. The gains hold for new questions in the same domains and, to a lesser degree, for entirely new domains like medicine and law. But training on one metacognitive task—rating a single answer—does not transfer to the other—choosing which of two answers is more likely correct—unless both are trained together. Multitask fine-tuning produces the broadest out-of-domain improvements, which matters for any deployment where a model must both report confidence and triage between competing answers.","feed_headline":"Fine-tuning makes LLM confidence match its accuracy","feed_subtitle":"Trained on self-consistency, models report better-calibrated confidence—even on unseen medical and legal questions.","key_machinery":"The central object is the consistency score s: for each question, 10 samples are drawn and s is the proportion matching the modal answer. An empirical mapping a(s) converts each consistency bin into a calibrated target confidence ĉ = a(s)+ε, where ε ∈ [−0.05, 0.05] regularizes against memorizing the 11 discrete consistency bins. Fine-tuning supervises the model to output ĉ as its numeric confidence for the modal answer; for pairwise comparison, training pairs questions with unequal consistency and asks the model to select the higher-s item. This mechanism converts sampling variability into a semantic-level confidence label that requires no internal token probabilities and only a single sampl","core_discovery":"The paper establishes that a model's verbalized confidence can be re-aligned with its true accuracy through supervised fine-tuning, using the model's own sampling consistency as the label source. After fine-tuning on general knowledge, math, and trivia questions, a baseline overconfident model shifts from confidence scores clustered near 0.85–0.95 to a spread that tracks empirical accuracy, cutting expected calibration error on within-domain held-out questions from 0.61 to 0.05 (MMLU-PRO) and raising discrimination AUC from 0.52 to 0.68. The improvements transfer to unseen medical and legal questions (e.g., MetaMedQA AUC 0.71→0.75 with single-task training, 0.71→0.79 with multitask training)","pith_inferences":["The use of self-consistency as the ground-truth signal means the model is trained to recognize its own sampling variability; a natural testable extension would be to fine-tune on correctness labels instead and compare whether metacognitive gains persist or actually improve accuracy.","The bidirectional failure of transfer suggests that verbalized confidence and relative comparison rely on distinct internal routines; a cheap diagnostic for shared representations would be to fine-tune on a third output format (e.g., confidence intervals or verbal 'low/medium/high') and see whether it bridges the gap.","The noise regularization ε is treated as a fixed ±0.05; it likely interacts with model scale and dataset size, and tuning it could yield further calibration gains at the low end of the consistency distribution, where the empirical mapping is noisiest.","Because the mapping a(s) is estimated on the training domains, out-of-domain gains are only as good as the transferability of the consistency–accuracy relationship; the paper's own TruthfulQA ECE increase is an early sign that this relationship is fragile across domains."],"forward_implications":["Calibrated verbalized confidence can be obtained from a single forward pass, avoiding the computational cost of sampling many outputs at inference time.","Fine-tuning on confidence estimation yields measurable gains in unseen domains such as medical reasoning (MetaMedQA AUC from 0.71 to 0.75), suggesting a route to safer LLM deployment in clinical settings.","Because accuracy is unchanged after fine-tuning, the discrimination improvements are genuinely metacognitive rather than a side effect of better answers.","Single-task fine-tuning on numeric confidence does not improve pairwise confidence ranking, so applications that rely on relative judgments (e.g., triage) need task-matched training data.","Multitask fine-tuning (confidence estimation plus pairwise comparison) produces the broadest cross-domain gains, e.g., MetaMedQA AUC 0.79 and LegalBench ECE 0.15, exceeding either task alone."],"fun_headline_variants":["Fine-tuning recalibrates LLM confidence to match accuracy","LLM confidence gets more truthful after fine-tuning on self-consistency","Fine-tuning teaches LLMs to know when they're right","Training on self-consistency improves LLM confidence calibration","LLM confidence aligns with accuracy after targeted fine-tuning"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The entire method leans on the assumption that the mapping from 'how often the model gives the same answer in 10 tries' to 'how likely that answer is right' measured in the training domains also holds in medicine, law, and truthfulness; if that relationship shifts, fine-tuned confidence will be miscalibrated in those new domains.","fun_headline_variants_meta":{"raw":{"variants":["Fine-tuning recalibrates LLM confidence to match accuracy","LLM confidence gets more truthful after fine-tuning on self-consistency","Fine-tuning teaches LLMs to know when they're right","Training on self-consistency improves LLM confidence calibration","LLM confidence aligns with accuracy after targeted fine-tuning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000718,"raw_usage":{"total_tokens":3102,"prompt_tokens":822,"completion_tokens":2280,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":566,"completion_tokens_details":{"reasoning_tokens":2197}},"tokens_in":566,"tokens_out":2280,"duration_ms":13165,"temperature":1.0,"reasoning_tokens":2197,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T13:26:58.804298+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct check: on an unseen benchmark, compute the empirical accuracy for each consistency bin and compare it with the mapping used to create training labels. If the bin-wise accuracies differ, a model fine-tuned with the transferred mapping should show no ECE improvement (or a reversal) relative to the baseline; the paper's TruthfulQA result—where ECE rose from 0.10 to 0.18 after fine-tuning—is a partial instance of that failure.","supporting_citations":[],"review_version":1}