{"id":"4fc7701a-1bbf-4770-bede-ff2d69baf674","arxiv_id":"2507.17951","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Larger pretrained language models show higher Bayesian Coherence Coefficient, meaning their in-context credence updates correlate more strongly with the likelihood-based updates Bayes' theorem would predict.","lead":"This paper measures whether language models update their probabilities in line with Bayes' rule by correlating predicted and observed log-odds changes, and finds that larger models score higher. The finding is a concrete behavioral correlation that could inform how we predict and steer larger AI systems.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"BCC is an affine-invariant correlation, so it cannot detect the systematic under-updating the paper reports; higher BCC may reflect higher signal-to-noise rather than closer adherence to Bayes' theorem.","rationale":"The paper is a transparent empirical study with a clear, falsifiable design and open code and data. I read the central claim as: BCC, a correlation of expected and observed log-updates, increases with scale, and this is evidence of more Bayesian-consistent belief updating. The load-bearing step is the leap from 'higher correlation' to 'more consistent with Bayes' theorem.' For each tuple, Bayes' theorem constrains Δ_observed and Δ_expected to be identical, not merely linearly related. Pearson correlation is invariant to affine transformations, so the actual constraint (slope = 1, intercept = 0) is invisible to BCC. The paper's own Table 1 shows all models under-update (gradient < 1); this is exactly the kind of deviation BCC cannot see. Thus the r = 0.906 result is compatible with a model that systematically multiplies evidence by 0.1 with small noise, which is not Bayesian. The reader's weakest assumption (token-probability proxy) is real, but this formal issue is more fundamental because it survives even if the proxy is accepted. I therefore agree with the reader's CONDITIONAL verdict but would add a mandatory reanalysis using metrics that penalize slope mismatch; if the scale result disappears, the abstract overclaims. No ad hominem is intended: the authors flag under-updating and limitations, which strengthens trust, but it does not remove the need for this check.","tokens_in":9921,"tokens_out":14922,"duration_ms":173774,"concrete_test":"Recompute the scale analysis on the released data with two alternative coherence metrics instead of BCC: (i) the OLS slope of Δ_observed on Δ_expected (Bayes requires slope 1), and (ii) standardized error mean((Δ_observed − Δ_expected)^2)/Var(Δ_expected). If the log-parameter correlations are substantially weaker than r = 0.906, or the model rankings disagree with BCC, the headline scaling is an artifact of using an affine-invariant correlation that ignores under-updating.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Equations (4)-(5) define Δ_observed and Δ_expected. Bayes' theorem requires Δ_observed = Δ_expected for every tuple, i.e., slope 1 and intercept 0. Equation (3) defines BCC as Pearson correlation, which is invariant to separate positive affine transformations of the two variables. Hence a model with Δ_observed = 0.1·Δ_expected + ε and arbitrarily small ε has BCC → 1, indistinguishable from a perfectly Bayesian model. The paper's Table 1 reports that every tested model has update gradient < 1, so the systematic deviation from Bayes is precisely the component BCC ignores. Higher BCC across scale may therefore be driven by larger models producing more discriminative expected updates (higher variance or signal-to-noise), not by smaller deviations from Bayes. This is a formal limitation of the metric, independent of whether cumulative token probabilities are a valid proxy for credence. The discussion and Appendix A acknowledge under-updating and metric choice, but the central scaling claim is still stated through BCC.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes the Bayesian Coherence Coefficient (BCC), defined as the Pearson correlation between expected log-likelihood-ratio updates and observed log-odds updates over a set of (class pair, evidence, history, category) tuples, and uses it to ask whether larger pretrained language models update propositional credences more consistently with Bayes' theorem. The authors generate a dataset of ten categories with classes, evidence snippets, and conversation histories, compute the relevant cumulative token probabilities with the tested models, and report BCC for models from five families (Falcon, Llama, Qwen, GPT-2, Pythia), a training-step analysis for Pythia, and correlations with six benchmarks. They find that BCC increases with log parameter count (r = 0.906, p < 10^-6) and with four of six benchmarks, and they report that all tested models under-update, with a regression gradient of observed on expected updates less than 1. The paper concludes that larger and more capable LLMs update their credences more consistently with Bayes' rule.","tokens_in":10151,"tokens_out":8308,"duration_ms":89834,"significance":"The question is timely and the paper is transparent: code and data are public, the evaluation spans multiple model families, and the authors explicitly acknowledge the token-probability proxy as unvalidated. The self-consistency design, in which expected and observed updates are computed from the same autoregressive model, is appropriate and is not circular. However, the central metric is insensitive to exactly the systematic deviation from Bayes' theorem that the paper itself documents: because BCC is a correlation, a model that outputs Δ_observed = 0.1 Δ_expected plus small noise receives BCC near 1, indistinguishable from a perfectly Bayesian model. The scaling claim therefore conflates linear association with Bayesian consistency. The paper is a useful empirical starting point, but the headline conclusion needs to be re-derived with a metric that penalizes deviations from the identity line, or explicitly reframed as a claim about linear association rather than closeness to Bayes.","major_comments":[{"comment":"The BCC is a Pearson correlation, which is invariant to separate positive affine transformations. Bayes' theorem requires Δ_observed = Δ_expected for every tuple, i.e., slope 1 and intercept 0 in the regression of Δ_observed on Δ_expected. The paper's own Table 1 reports that every model has an update gradient less than 1, so the systematic under-updating is invisible to BCC. Indeed, a model with Δ_observed = 0.1 Δ_expected + ε and arbitrarily small noise has BCC → 1, indistinguishable from a perfectly Bayesian model. Thus the main scaling result (r = 0.906, p < 10^-6) supports the statement that larger models' updates are more linearly predictable from the likelihood ratios, not that they are closer to the Bayes-optimal update. This is load-bearing for the abstract and Section 5 claims. I recommend reporting a metric that directly measures closeness to the identity line, such as a normalized mean squared error after matching variances, or the regression slope and intercept with their uncertainties, and presenting scaling results for those quantities.","section":"Section 2, Eqs. (3)–(5); Table 1"},{"comment":"The paper explicitly states in Section 6 that 'we use cumulative token probabilities as a proxy for credence in the proposition they compose' and that 'It is unclear whether this is an accurate proxy for action-relevant belief states.' Because every BCC value and every update gradient is computed from these text-generation probabilities, the empirical findings may characterize coherence of next-token probabilities rather than agent beliefs. This is not a circularity, since the test is a self-consistency test, but it is an unvalidated and load-bearing assumption. The paper should validate the proxy on a subset of items, for example by comparing with probabilities elicited through classification or betting tasks, or it should state clearly that the conclusions concern token-probability coherence only.","section":"Section 6; Eqs. (4)–(5)"},{"comment":"The reported p-values for the scaling correlations are not reliable. The BCC for each model is computed over 6,460 tuples that are strongly non-independent because multiple class pairs, evidence items, and histories share the same category and the same model draws; p-values that treat each tuple as an independent observation are therefore far too small. Conversely, the correlation with log parameter count has only one point per model (roughly 10–16 models total), and p < 10^-6 is extremely sensitive to the exact number of models and to family membership; a mixed-effects or permutation analysis should be used. The six benchmark correlations are also not corrected for multiple comparisons. The manuscript should report confidence intervals, the exact number of models, and block-bootstrap or permutation tests that account for the dependence structure.","section":"Section 4; Fig. 3; Fig. 5"}],"minor_comments":[{"comment":"The phrase 'where c1,c 2∈ C are pairs of classes' should read 'where c1 and c2 are distinct classes in C.'","section":"Section 2, Eq. (2)"},{"comment":"The statement that 'p values were too small to be properly rendered' should be replaced by an explicit bound, such as p < 10^-16, so that readers know the magnitude.","section":"Figure 2 caption"},{"comment":"The dataset is generated by GPT-4o through the ChatGPT interface, and the paper does not discuss whether the generation model's idiosyncrasies affect the resulting BCC values or provide a sensitivity analysis to the generation procedure.","section":"Section 3"},{"comment":"The desiderata require at least five classes per category, while the JSON schema sets minItems to 2; this inconsistency should be resolved.","section":"Appendix C.1 and C.3"},{"comment":"The claim that larger models show an update gradient closer to 1 is reported descriptively in Table 1 but is not accompanied by a significance test or confidence interval; these quantities should be reported along with the BCC results.","section":"Section 5, Table 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is likely fixable by re-analysis, but the current formulation of the central claim is not supported by the BCC metric. The published code and data make such re-analysis feasible, so I suggest encouraging the authors to revise rather than rejecting outright. The novelty concern is modest but acceptable; the main issue is the mismatch between the metric and the claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing to know: this paper introduces a new metric (BCC, a correlation between expected and observed log-odds updates) and a new dataset for testing whether LLM credences update in a Bayes-consistent way. The headline finding is that BCC increases log-linearly with model size (r = 0.906, p < 10^-6) across five families, and about half the benchmark correlations are significant. The dataset and code are public, the methods are transparent, and the limitations section is admirably honest about the token-probability-as-credence proxy and the under-updating they observe. That is real, reproducible work and it deserves credit.\n\nThe main soft spot is the metric itself. BCC is a Pearson correlation, and correlations are invariant to affine transformations. Bayes' theorem demands not just correlation but equality: observed update = expected update, slope 1, intercept 0. Their Table 1 shows every model has update gradient well below 1, so every model systematically under-updates. A model with observed = 0.1 * expected + noise would get BCC near 1 as long as the noise is small. That means high BCC does not imply close adherence to Bayes' rule; it can just mean the expected updates are discriminative and the observed updates track them with some gain below 1. The stress-test note is right: the scaling result may partly reflect larger models having higher signal-to-noise in their token probabilities, not smaller deviations from Bayes. This is a formal limitation of BCC, independent of the credence proxy.\n\nThe other soft spots are secondary but real. The p-values are computed over 6,460 tuples that are not independent (class pairs within a category share structure), and the scaling correlation has only 10 model points. There are no confidence intervals on BCC. None of this is fatal, but it makes the very small p-values misleading. And the token-probability proxy for credence is load-bearing; the authors flag it as unvalidated, which I respect, but it means the whole exercise measures coherence of text probabilities, not necessarily of action-relevant beliefs.\n\nHow much does this change my verdict? The central correlation finding is plausible as a measurement of something, but the jump from 'higher BCC' to 'more coherent with Bayes' theorem' is too strong. The authors do acknowledge under-updating, but they still state the scaling claim through BCC. The paper would be a solid contribution if reframed as 'larger models show higher linear agreement between expected and observed token-probability updates, with a consistent under-updating bias that shrinks slightly with scale.' That is still an interesting result, but it is not the same as Bayesianity.\n\nWho should read it? Anyone working on consistency checks for LLM forecasting or world-model evaluation. It is a useful counterpoint to Fluri et al.'s null scaling result, and the dataset could be reused. It does not settle the question, but it is a serious empirical effort with clear limitations.\n\nRecommendation: send it to peer review. The metric issue is fixable in framing, and the empirical pattern deserves scrutiny. A good referee would push for confidence intervals, independence-aware statistics, and a reanalysis using slope or an error metric alongside BCC. With those changes, this could be a genuinely valuable paper.\n\nBest,\n[You]","headline":"A useful, honest new measurement of LLM Bayesian coherence with a real scaling trend, but the headline metric ignores the systematic under-updating that the paper itself reports, so the 'more Bayesian' framing needs qualification.","tokens_in":10642,"tokens_out":1318,"would_cite":false,"duration_ms":16112,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Larger language models update their beliefs about propositions more consistently with Bayes' theorem, with the Bayesian Coherence Coefficient rising roughly log-linearly with model scale.","keywords":["Bayesian Coherence Coefficient","large language models","belief updating","model scaling","in-context learning","credence","consistency metrics","AI safety"],"falsifier":"Ask a model to place bets or set prices on the candidate classes before and after each piece of evidence, and compute the same coherence coefficient from those decision-based odds updates. If the decision-based measure diverges from the token-probability BCC, or if its scaling trend with model size disappears, the paper's conclusion would be about text-generation probabilities rather than beliefs.","tokens_in":9664,"feed_emoji":"📈","tokens_out":12675,"duration_ms":119975,"temperature":0.7,"pith_summary":"The paper asks whether language models update their beliefs about propositions in a way consistent with Bayes' theorem when evidence appears in the conversation. It introduces a Bayesian Coherence Coefficient (BCC), defined as the correlation between the expected log likelihood ratio of evidence under two candidate classes and the model's observed log odds update from prior to posterior, with all probabilities estimated by cumulative token probabilities. Across five pretrained model families, BCC increases approximately log-linearly with parameter count ($r = 0.906$, $p < 10^{-6}$) and correlates significantly with performance on four of six common benchmarks. The authors take this as early evidence that larger, more capable models learn more Bayes-consistent belief updates, which they argue matters for interpretability, steerability, and the safety risks of systems that begin to resemble expected-utility maximizers.","feed_headline":"Larger language models update beliefs in a more Bayesian way","feed_subtitle":"A coherence score rises with model size and benchmark ability, suggesting scale improves in-context belief updates.","key_machinery":"The load-bearing object is the Bayesian Coherence Coefficient (BCC), a correlation between expected and observed log-odds updates. For a pair of candidate classes $(c_1,c_2)$, evidence $x$, conversation history $h$, and category $k$, the expected update is the log likelihood ratio $\\log [P_\\theta(x|c_1,h,k)/P_\\theta(x|c_2,h,k)]$, and the observed update is the log posterior ratio minus the log prior ratio, $\\log [P_\\theta(c_1|x,h,k)/P_\\theta(c_2|x,h,k)] - \\log [P_\\theta(c_1|h,k)/P_\\theta(c_2|h,k)]$. All four probabilities are read from the model as cumulative conditional token probabilities of the class or evidence strings, with separate model instances used for priors, likelihoods, and posteriors. Because BCC is a correlation rather than an error metric, it does not reward flat, high-entropy output distributions the way the authors show an error-based alternative does; the metric therefore isolates whether updates move in the right direction and by roughly the right log-odds amount.","core_discovery":"The paper's central claim is that pre-trained language models become more coherent Bayesian updaters as they get larger: across five model families, the Bayesian Coherence Coefficient (BCC)—the correlation, over 6,460 (class pair, evidence, history, category) tuples, between the expected update $\\log [P_\\theta(x|c_1,h,k)/P_\\theta(x|c_2,h,k)]$ and the observed update $\\log [P_\\theta(c_1|x,h,k)/P_\\theta(c_2|x,h,k)] - \\log [P_\\theta(c_1|h,k)/P_\\theta(c_2|h,k)]$—increases approximately log-linearly with parameter count ($r = 0.906$, $p < 10^{-6}$). Every tested model scores above zero, meaning even small models update in the right direction more often than a random policy, and the fitted gradient of observed versus expected updates is below 1 for all models, so models systematically under-update their credences rather than over-update them. Four of six benchmark correlations (BIG-Bench Hard, GPQA, MMLU-PRO, Math Lvl 5) are significantly positive; IFEval and MUSR are positive but not significant. The authors present this as evidence that scale and capability make in-context belief updates more Bayes-consistent, and they note the result must be reconciled with an earlier study that found no such improvement.","pith_inferences":["An untested but natural next experiment is to compute BCC from decision-based credences, such as prices or bets the model is prompted to place on each class before and after evidence, and compare this with the token-probability BCC; divergence would mean the paper's metric tracks text probabilities rather than beliefs.","The dataset and metric could be applied to instruction-tuned and reinforcement-learning-tuned models at fixed parameter counts, which the paper explicitly leaves to future work; it is open whether alignment training raises, lowers, or leaves BCC unchanged.","The positive correlations with general benchmarks raise the possibility that BCC is capturing general reasoning ability rather than specifically Bayesian updating; synthetic evidence with researcher-specified likelihood ratios would isolate the mechanism.","If BCC is accepted as a belief-coherence proxy, it could serve as a cheap monitoring signal during training or scaling: a drop in BCC even as benchmarks improve would flag a model whose in-context beliefs are becoming less coherent."],"forward_implications":["If the trend continues beyond the tested scale, larger pretrained models should be increasingly predictable and steerable by agents that share their world model, because their belief revisions move closer to the Bayesian ideal.","More coherent Bayesian updating makes it harder to conceal information from a model and harder to evaluate it without the model conditioning on the fact that it is being evaluated, since subtle cues in the evidence become inferable.","If coherent beliefs are paired with coherent preferences, future systems move closer to expected-utility maximizers, which brings clearer goal specification but also the risk of misaligned optimization, incorrigibility, and power seeking.","The systematic under-updating gradient (below 1) means all tested models move their credences less than full Bayesian updating demands; the authors connect this to the evidence strings being much less likely than the class strings, so datasets with more probable evidence would likely change the observed gradient.","Because BCC is a correlation, it is not confounded by the entropy of a model's output distribution, so it can compare coherence across models that differ widely in confidence."],"supporting_citations":[{"why":"Motivates the premise that next-token prediction can implement constrained Bayesian updates on structured data, making a coherence check over propositions worth running.","marker":"Piotrowski et al., 2025"},{"why":"Provides the earlier negative scaling result for Bayesian consistency that this paper must explain or contrast with its positive BCC scaling.","marker":"Fluri et al., 2023"},{"why":"Documents that larger models violate logical and probabilistic consistency checks less often, the scaling pattern this paper extends to belief-update coherence.","marker":"Paleka et al., 2025"},{"why":"Shows larger models exhibit higher preference coherence, supporting the broader empirical pattern that consistency grows with model size.","marker":"Mazeika et al., 2025"},{"why":"Supplies the Pythia checkpoints used to measure how BCC changes with training steps.","marker":"Biderman et al., 2023"},{"why":"Source of the parameter counts and benchmark scores correlated with BCC.","marker":"Fourrier et al., 2024"}],"fun_headline_variants":["Scale makes LLM belief updates more Bayesian","Bigger models, more Bayesian belief updates","LLM size predicts Bayesian coherence in updates","Larger pre-trained LLMs update beliefs more Bayes-consistently"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything rests on treating the probability a model assigns to the words of a claim as the strength of its belief in that claim; the paper itself states that it is unclear whether these token probabilities reflect an action-relevant belief state.","fun_headline_variants_meta":{"raw":{"variants":["Scale makes LLM belief updates more Bayesian","Bigger models, more Bayesian belief updates","LLM size predicts Bayesian coherence in updates","Larger pre-trained LLMs update beliefs more Bayes-consistently"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000622,"raw_usage":{"total_tokens":2881,"prompt_tokens":944,"completion_tokens":1937,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":560,"completion_tokens_details":{"reasoning_tokens":1876}},"tokens_in":560,"tokens_out":1937,"duration_ms":13256,"temperature":1.0,"reasoning_tokens":1876,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:16:57.122732+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Ask a model to place bets or set prices on the candidate classes before and after each piece of evidence, and compute the same coherence coefficient from those decision-based odds updates. If the decision-based measure diverges from the token-probability BCC, or if its scaling trend with model size disappears, the paper's conclusion would be about text-generation probabilities rather than beliefs.","supporting_citations":[{"cited_title":"Open LLM Leaderboard 2, 2024","cited_arxiv_id":null,"evidence_quote":"Source of the parameter counts and benchmark scores correlated with BCC."}],"review_version":1}