Pith Number

pith:UN2O3Z3Z

pith:2026:UN2O3Z3ZNARDLNSIVDWXADDW6J

not attested not anchored not stored refs pending

LLMs are not (consistently) Bayesian: Quantifying internal (in)consistencies of LLMs' probabilistic beliefs

Adam Goli\'nski, Chacha Chen, Guillermo Sapiro, Masha Fedzechkina, Matthew J\"orke, Nicholas Foti, Sinead Williamson

Large language models do not consistently update probabilistic beliefs according to Bayesian rules.

arxiv:2605.06915 v2 · 2026-05-07 · cs.LG

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{UN2O3Z3ZNARDLNSIVDWXADDW6J}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

Some of these approaches produce (nearly) Bayesian updates; others seem to use a learned heuristic. Surprisingly, the non-Bayesian heuristic updates often outperform exact Bayesian computation in terms of downstream task performance -- indicating the LLMs' probabilistic models of the world are misspecified.

C2weakest assumption

That LLMs possess stable internal probabilistic beliefs that can be reliably elicited and compared to Bayesian standards via prompting or other interfaces, and that the introduced information processing gap accurately quantifies internal inconsistencies rather than surface-level response artifacts.

C3one line summary

LLMs do not consistently perform Bayesian updates on probabilistic beliefs; heuristic approaches often outperform exact Bayesian computation on downstream tasks, indicating misspecified internal models of the world.

Formal links

2 machine-checked theorem links

Receipt and verification

First computed	2026-05-28T01:04:41.768137Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

a374ede779682235b648a8ed700c76f246378e96401b2db890275350035f5bfe

Aliases

arxiv: 2605.06915 · arxiv_version: 2605.06915v2 · doi: 10.48550/arxiv.2605.06915 · pith_short_12: UN2O3Z3ZNARD · pith_short_16: UN2O3Z3ZNARDLNSI · pith_short_8: UN2O3Z3Z

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/UN2O3Z3ZNARDLNSIVDWXADDW6J \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: a374ede779682235b648a8ed700c76f246378e96401b2db890275350035f5bfe

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "9a640245a89cd0475aa1110ca504e1892737855942fd84819109d3f8d75cab0d",
    "cross_cats_sorted": [],
    "license": "http://creativecommons.org/licenses/by-nc-nd/4.0/",
    "primary_cat": "cs.LG",
    "submitted_at": "2026-05-07T20:25:02Z",
    "title_canon_sha256": "7fb1dafc4e1383768a754c89e1a14721f6c39c3d5acaf156f3369206f082bc50"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2605.06915",
    "kind": "arxiv",
    "version": 2
  }
}