{"id":"12b8b66c-c281-4e93-ab74-0251a76de733","arxiv_id":"2504.13644","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"LLMs frequently violate the probability axioms of complementarity and monotonicity when rating the truth of everyday claims, even when using chain-of-thought or argumentative prompting techniques.","lead":"The paper presents a new dataset of 517 everyday claims with logical variants and asks seven large language models to rate how likely each claim is to be true. It finds that the models frequently assign probabilities that break basic rules of probability, such as a claim and its negation summing to more or less than 100%.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Monotonicity scores count equality as a violation despite intentionally subtle claim modifications and integer percent outputs; reported violation rates may be inflated.","rationale":"I agree with the reader's identification of the strict-inequality assumption as the weakest point of the paper. The concern is load-bearing because it directly affects the quantitative violation rates that support the central claim. The paper's own design choices make the assumption fragile: the modifications are intentionally close in probability, and the integer-percent response format forces ties for any pair whose true probabilities differ by less than a percentage point. The paper neither reports tie frequencies nor runs a sensitivity analysis that counts ties as non-violations. My proposed re-scoring is a direct, low-cost check of whether the strictness assumption changes the conclusions. If violation rates remain high after allowing ties, the empirical finding is robust; if they drop, the central claim must be softened. Even in the worst case, the violation rates are large enough that the core phenomenon likely persists, but the exact magnitude is uncertain. The reader's CONDITIONAL verdict therefore remains appropriate: the authors should either justify the strictness assumption with data or report both strict and non-strict scoring, and soften the claim that LLMs 'lack the ability' to reason probabilistically.","tokens_in":9676,"tokens_out":8700,"duration_ms":80916,"concrete_test":"Re-score all specialisation and generalisation evaluations from Tables 1 and 3 using the correct non-strict inequality as the criterion: a specialisation is a violation only if P(specialised) > P(original), and a generalisation is a violation only if P(generalised) < P(original); any equality is counted as correct. Compare the resulting accuracies with the published ones. If any entry's accuracy increases by more than 5 percentage points, the strictness assumption is materially load-bearing, and the reported violation rates need to be reinterpreted with that caveat. If accuracies are essentially unchanged, the concern is minor.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.1 scores monotonicity with strict inequalities: P(original) > P(specialised) and P(original) < P(generalised), so any equality in the model's estimates is counted as a violation. The paper justifies strictness by asserting that 'the practical instantiation of our dataset means that the difference between the modified and original claim are always non-zero' (Section 3.1), but this rests only on manual inspection, not on any formal check. Two features of the design make the strictness assumption fragile. First, Section 2 states that specialised and generalised claims are deliberately generated to be 'not too distant (in a probabilistic sense)' from the original, meaning the true probability gaps may be small. Second, the models output integer percentage scores, and the final estimate is the mean of only 5 samples; a rational model facing a true gap of, say, 0.5% can easily produce equal rounded means. The paper does not report how often the estimated probabilities for a pair are exactly equal, so the contribution of these ties to the 50-70% monotonicity violation rates is unknown. If a substantial fraction of the violations are ties, the central claim of frequent incoherence is correspondingly weakened, though the high rates make it unlikely that ties alone explain the finding.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript introduces the Rational Probabilistic Belief (RPB) dataset of 517 base claims with indeterminate truth values, each accompanied by a complement, a more specific version, and a more general version. It then evaluates seven LLMs with several uncertainty-quantification methods, including direct prompting, chain-of-thought, an argumentative LLM approach, and top-k logit sampling, measuring how often the models violate complementarity and monotonicity. The authors report high violation rates across models and conclude that current LLMs frequently fail to provide rational, coherent probabilistic beliefs even when enhanced with uncertainty-quantification techniques.","tokens_in":9870,"tokens_out":7432,"duration_ms":71459,"significance":"If the central finding holds, the paper provides a useful negative result for a community that increasingly relies on LLM-generated probability estimates in decision-support systems. The contribution is strengthened by the public release of code and data, by the evaluation of multiple open- and closed-source models, and by the use of simple, externally motivated normative constraints rather than a fitted benchmark. The paper also explicitly separates objective uncertainty from subjective model uncertainty, which is a helpful conceptual clarification. The main risk is that the monotonicity metric treats strictness in a way that may inflate violation rates, and this needs to be addressed before the quantitative claims can be taken at face value.","major_comments":[{"comment":"The monotonicity evaluation defines violations via the strict inequalities P(original) > P(specialised) and P(original) < P(generalised), so an exact equality in the estimated probabilities is counted as a violation. The justification given in Section 3.1, that the dataset's practical instantiation ensures the difference is always non-zero, rests on manual inspection rather than on a formal guarantee. The problem is amplified by two design choices: Section 2 states that specialised and generalised claims are deliberately generated to be not too distant probabilistically from the original, and the models output integer percentage scores whose final estimate is the mean of only five samples. A rational model facing a true gap of a fraction of a percent can therefore easily produce equal rounded means. The paper does not report how often the estimated probabilities are exactly equal for a pair, so the contribution of these ties to the 50-70% monotonicity violation rates is unknown. The authors should report the tie rate and either recompute violation rates treating ties as non-violations, or provide independent evidence, such as a documented annotation study or adjustment log, that every specialised claim is strictly less probable and every generalised claim strictly more probable than its base claim. This is load-bearing because the monotonicity violation rates are a central pillar of the paper's conclusion.","section":"§3.1, §2"},{"comment":"The complementarity deviation is printed as |1 - P(original statement) + P(negated statement)| * 100. The intended quantity is clearly |1 - (P(original statement) + P(negated statement))| * 100. As written, the formula would measure |1 - P(original) + P(negated)|, which is not the deviation from P(A)+P(A^c)=1. Please correct the formula and verify that the values reported in Table 2 are computed from the intended expression. Because complementarity is one of the two main empirical claims, this ambiguity is a reproducibility issue that must be fixed.","section":"Table 2 caption, §3.2"},{"comment":"The construction of the RPB dataset relies on a manual check in which the authors state, 'We adjust any samples that were found to violate either of these requirements.' The paper does not specify the adjustment criteria, the number or fraction of samples adjusted, or whether the adjustments were performed by more than one annotator. Since the validity of the specialised and generalised variants is what makes the monotonicity comparisons meaningful, this step should be documented in detail, and the final data, including an adjustment log if possible, should accompany the released dataset.","section":"§2"}],"minor_comments":[{"comment":"The words 'former' and 'latter' appear to be swapped relative to the listed principles complementarity and monotonicity: the sentence beginning 'For the former' describes specialisation and generalisation, which belong to monotonicity, while the sentence beginning 'For the latter' describes the negated claim, which belongs to complementarity.","section":"§3, first paragraph"},{"comment":"The tables report proportions without confidence intervals or significance tests. Given that each estimate is the mean of only five samples, the precision of the violation rates should be quantified, for example with bootstrap confidence intervals.","section":"Tables 1-3"},{"comment":"The validation set and prompt-selection procedure are described, but no validation results are reported, so the reader cannot assess whether the chosen direct-prompting candidate is indeed best among the four candidates.","section":"§4.1"},{"comment":"The top-k logit method discards probability mass outside the top five tokens; the possible bias from this truncation should be acknowledged or quantified in the discussion of the method's results.","section":"§4.4"},{"comment":"The dataset was generated by GPT-4o, which is also one of the evaluated models. Manual filtering mitigates some concerns, but the potential dependence of the results on the generator's own biases should be discussed as a limitation.","section":"§2, §5.1"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. First, the paper contributes a genuinely useful dataset—517 base claims with complements, specialisations, and generalisations—and evaluates seven models across four uncertainty-quantification methods, with code and data released. Second, the headline claim that LLMs \"lack the ability\" to hold rational probabilistic beliefs is stronger than the evidence, and the monotonicity results rest on an assumption that is asserted, not proven.\n\nThe complementarity findings are the cleanest: models routinely give P(A) + P(not A) far from 1, and that holds across models and methods. This is a real empirical result and worth having. The monotonicity results are more fragile. Section 3.1 scores strict inequalities, P(original) > P(specialised) and P(original) < P(generalised), justified by saying the dataset's variants always differ in probability. That claim is supported only by manual inspection of GPT-4o-generated claims. The paper even says the modifications are deliberately \"not too distant\" from the original, and models output integer percentages as the mean of five samples. Equal rounded estimates can easily occur even when true probabilities differ. The paper never reports how many violations are ties. A substantial tie rate would inflate the headline numbers. This needs fixing—the violation rates are high enough that ties are unlikely to explain everything, but I'd want tie counts and a sensitivity check before trusting the precise figures.\n\nOther soft spots are minor. No baselines or significance tests, so we don't know how these rates compare to random or uniform guessing. The abstract's \"lack the ability\" should be softened to something like \"frequently violate.\" Table 2's formula is missing parentheses around the sum, and its caption calls the measure deviation from ground truth when it's actually deviation from the complementarity constraint. Also, the dataset was generated by GPT-4o, one of the evaluated models—not fatal, but worth a robustness note.\n\nWho the paper is for: anyone working on LLM uncertainty quantification, forecasting with LLMs, or neurosymbolic reasoning. It deserves a serious referee. I'd send it to peer review and ask for the monotonicity analysis to be redone with tie counts, a clearer justification of strictness, and some baselines. The core phenomenon is likely real; the paper just needs to align its claims with its evidence.","headline":"Useful dataset and broad evaluation of LLM probabilistic coherence, but the headline claim overreaches and the monotonicity metric needs a tie analysis before the numbers are trustworthy.","tokens_in":10392,"tokens_out":2656,"would_cite":true,"duration_ms":24862,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper shows that current large language models frequently violate the standard probability axioms of complementarity and monotonicity when assigning confidence to natural-language claims, and that uncertainty-quantification prompts do…","keywords":["probabilistic reasoning","large language models","uncertainty quantification","complementarity","monotonicity","objective probability","probability axioms","natural language claims"],"falsifier":"Have independent human annotators label a random sample of the RPB specialised and generalised pairs as strictly more probable, strictly less probable, or roughly equal, and recompute the monotonicity violation rates using only pairs where the strict inequality is uncontested; if the rates fall to near zero the central result would collapse, and if they stay high the result would survive.","tokens_in":9429,"feed_emoji":"🎲","tokens_out":14008,"duration_ms":111334,"temperature":0.7,"pith_summary":"Large language models are increasingly asked to give probabilities for ordinary claims, but this paper asks whether those estimates are coherent in the sense required by probability theory. It builds a dataset of 517 natural-language claims with indeterminate truth values, each paired with a negation, a more specific version, and a more general version, and compares model estimates against two corollaries of the standard probability axioms: complementarity ($P(A)+P(A^c)=1$) and monotonicity (specialising a claim cannot raise its probability). Across seven open- and closed-source models and four uncertainty-quantification methods, the models frequently violate both properties, and larger models only reduce, not remove, the errors. The paper concludes that current LLMs cannot be trusted to supply coherent probabilistic beliefs and that better elicitation prompts are not the cure.","feed_headline":"LLMs violate basic probability axioms on ordinary claims","feed_subtitle":"Even chain-of-thought and argumentative prompts leave probability estimates incoherent.","key_machinery":"The paper's machinery is a consistency test built from three corollaries of the standard probability axioms: complementarity, $P(A)+P(A^c)=1$; specialisation monotonicity, $P(A')<P(A)$ whenever $A' \\subset A$; and generalisation monotonicity, $P(A)<P(A')$ whenever $A \\subset A'$. The Rational Probabilistic Beliefs dataset instantiates these patterns in natural language: 517 base claims, each with a complement, a more specific variant, and a more general variant (2068 samples in total). The evaluation elicits a probability for each variant using four methods, namely direct prompting, chain-of-thought prompting, a structured argumentative LLM, and top-k logit sampling, and then checks whether the elicited numbers satisfy the equalities and strict inequalities. This turns abstract axiomatic constraints into a measurable benchmark.","core_discovery":"On the paper's own terms, the central discovery is a negative one: state-of-the-art LLMs do not maintain rational probabilistic beliefs about claims with indeterminate truth values. Accuracies on the specialisation monotonicity task range from roughly 45% to 69% across models and elicitation methods, and on generalisation from roughly 51% to 84%, so a large share of elicited probability triples are internally inconsistent. Complementarity scores show similar trouble: for most models, over 70% of responses deviate from $P(\\text{claim}) + P(\\text{negation}) = 1$ by more than 5 points, and the deviations are not eliminated by chain-of-thought, argumentative scaffolding, or top-k logit sampling. The paper reads these results as evidence that capability at language and even calibration at the level of single questions do not imply coherent probabilistic reasoning, and it points to neurosymbolic systems, with a symbolic module enforcing consistency, as the more promising route.","pith_inferences":["The paper does not test whether rephrasing the same claim in different wording changes the elicited probability; measuring that gap would separate genuine axiom violations from sensitivity to surface phrasing.","Because the dataset was generated by GPT-4o and the same model family is among the tested systems, evaluating on independently human-authored claims would show whether the failures are intrinsic to current LLMs or partly an artefact of self-generation.","If the result holds across human-authored claims, a practical consequence is that asking an LLM for a single number is the wrong interface: better to have it produce structured evidence and let a symbolic probabilistic reasoner combine it into a coherent distribution."],"forward_implications":["Any automated decision system that consumes LLM confidence scores as probabilities will inherit the incoherence: a more specific version of a claim can receive a higher probability than the claim it implies.","Uncertainty-quantification add-ons such as chain-of-thought prompting, argumentative scaffolding, and logit averaging do not restore axiom compliance, so prompt engineering alone is not a fix.","The pattern across model sizes suggests that scaling alone will not make LLMs reliable probabilistic reasoners.","Systems that need coherent probabilities should combine LLM natural-language processing with a symbolic module that enforces constraints such as $P(A)+P(A^c)=1$, as the paper itself suggests."],"supporting_citations":[{"why":"Gives the premise that a rational agent must not violate the axioms of probability, which the paper turns into its evaluation criterion.","marker":"Bas 2019"},{"why":"Supplies the probability axioms from which complementarity and the two monotonicity properties are derived as corollaries.","marker":"Kolmogorov 1963"},{"why":"Defines objective probability as what a perfectly rational agent with perfect past information would assign, setting the target for the measured estimates.","marker":"Rayo 2019"},{"why":"Provides both the direct-prompting template and the argumentative LLM method used to elicit probability estimates.","marker":"Freedman et al. 2024"},{"why":"Introduces the 'Let's think step by step' instruction used as the chain-of-thought elicitation prompt.","marker":"Kojima et al. 2022"},{"why":"Supplies the 4-bit quantisation applied to all open-source models, tying the experimental results to a specific implementation detail.","marker":"Dettmers et al. 2023"}],"fun_headline_variants":["LLMs fail probabilistic coherence on everyday claims","AI language models can't keep probability sums to one","LLMs show irrational probability beliefs in new tests","Even advanced LLMs stumble on basic probability rules","Large language models lack rational probability reasoning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The monotonicity result assumes that every GPT-4o-generated 'more specific' or 'more general' variant is strictly less or strictly more probable than the base claim, an assumption the paper supports only by manual inspection of the generated variants.","fun_headline_variants_meta":{"raw":{"variants":["LLMs fail probabilistic coherence on everyday claims","AI language models can't keep probability sums to one","LLMs show irrational probability beliefs in new tests","Even advanced LLMs stumble on basic probability rules","Large language models lack rational probability reasoning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000225,"raw_usage":{"total_tokens":1415,"prompt_tokens":847,"completion_tokens":568,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":463,"completion_tokens_details":{"reasoning_tokens":499}},"tokens_in":463,"tokens_out":568,"duration_ms":5466,"temperature":1.0,"reasoning_tokens":499,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:02:45.876900+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have independent human annotators label a random sample of the RPB specialised and generalised pairs as strictly more probable, strictly less probable, or roughly equal, and recompute the monotonicity violation rates using only pairs where the strict inequality is uncontested; if the rates fall to near zero the central result would collapse, and if they stay high the result would survive.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the premise that a rational agent must not violate the axioms of probability, which the paper turns into its evaluation criterion."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the probability axioms from which complementarity and the two monotonicity properties are derived as corollaries."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines objective probability as what a perfectly rational agent with perfect past information would assign, setting the target for the measured estimates."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides both the direct-prompting template and the argumentative LLM method used to elicit probability estimates."},{"cited_title":"S.; Reid, M.; Matsuo, Y .; and Iwa- sawa, Y","cited_arxiv_id":null,"evidence_quote":"Introduces the 'Let's think step by step' instruction used as the chain-of-thought elicitation prompt."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the 4-bit quantisation applied to all open-source models, tying the experimental results to a specific implementation detail."}],"review_version":1}