The reviewed record of science sign in
Pith

arxiv: 2503.04722 · v2 · pith:MMITW4YH · submitted 2025-03-06 · cs.CL · cs.AI· cs.LG

Enough Coin Flips Can Make LLMs Act Bayesian

Ritwik Gupta , Rodolfo Corona , Jiaxin Ge , Eric Wang , Dan Klein , Trevor Darrell , David M. Chan This is my paper

Reviewed by Pithpith:MMITW4YHopen to challenge →

classification cs.CL cs.AIcs.LG
keywords llmsbayesianbiasedcoinflipspriorsin-contextupdates
0
0 comments X
read the original abstract

Large language models (LLMs) exhibit the ability to generalize given few-shot examples in their input prompt, an emergent capability known as in-context learning (ICL). We investigate whether LLMs use ICL to perform structured reasoning in ways that are consistent with a Bayesian framework or rely on pattern matching. Using a controlled setting of biased coin flips, we find that: (1) LLMs often possess biased priors, causing initial divergence in zero-shot settings, (2) in-context evidence outweighs explicit bias instructions, (3) LLMs broadly follow Bayesian posterior updates, with deviations primarily due to miscalibrated priors rather than flawed updates, and (4) attention magnitude has negligible effect on Bayesian inference. With sufficient demonstrations of biased coin flips via ICL, LLMs update their priors in a Bayesian manner.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Beyond Facts: Benchmarking Distributional Reading Comprehension in Large Language Models

    cs.CL 2026-03 conditional novelty 7.0

    Text2DistBench is a new scalable benchmark showing LLMs outperform random baselines on distributional reading comprehension from YouTube comments but vary widely by question type and distribution characteristics.

  2. Flipping Against All Odds: Reducing LLM Coin Flip Bias via Verbalized Rejection Sampling

    cs.LG 2025-06 unverdicted novelty 7.0

    Verbalized Rejection Sampling reduces bias in LLM Bernoulli sampling by prompting the model to reason about and accept or reject proposed samples.

  3. Pre-trained Large Language Models Learn Hidden Markov Models In-context

    cs.LG 2025-06 unverdicted novelty 7.0

    Pre-trained LLMs learn to predict HMM-generated sequences via in-context learning, approaching theoretical optimum on synthetic HMMs and matching expert models on real animal decision data.

  4. EviCare: Enhancing Diagnosis Prediction with Deep Model-Guided Evidence for In-Context Reasoning

    cs.CL 2026-04 unverdicted novelty 5.0

    EviCare uses deep model-guided evidence to enhance LLM in-context reasoning for accurate diagnosis prediction from EHRs, outperforming baselines by 20.65% on average and 30.97% for novel diagnoses on MIMIC datasets.