Pith. sign in

REVIEW 4 major objections 5 minor 26 references

Using Large Language Models to Suggest Informative Prior Distributions in Bayesian Statistics

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLMs reliably identify the direction of statistical associations but fail to calibrate the strength of their suggested Bayesian priors, which tend to be overconfident, according to tests on heart-disease and concrete-strength data.

desk verdict An honest, modest comparison paper whose qualitative direction-recall and overconfidence findings are worth taking seriously, but whose quantitative LLM ranking should not be believed at face value. read the letter →

arxiv 2506.21964 v1 pith:55COL3QN submitted 2025-06-27 stat.ME cs.AIcs.CL

Michael A. Riegler , Kristoffer Herland Hellton , Vajira Thambawita , Hugo L. Hammer This is my paper
classification stat.MEcs.AIcs.CL MSC 62F1562B1062J12
keywords largelanguagemodelsBayesianstatisticspriorelicitationinformativepriorsKullback-Leiblerdivergencemaximumlikelihoodestimatorlogisticregressionoverconfidence
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Bayesian analysis often settles for vague priors because turning domain knowledge into an actual distribution is difficult, subjective, and time-consuming. This paper tests whether a large language model can do that translation, using a detailed prompt that makes the model justify each hyperparameter, propose both moderately and weakly informative sets, and assign confidence scores. Across a heart-disease logistic regression and a concrete-strength linear regression, the three tested models—Claude, Gemini, and ChatGPT—all identified the correct direction for every association, such as males having higher heart-disease risk. Yet their moderately informative priors, the ones the models themselves trusted most, were often overconfident and sat far from the data, while the weakly informative priors generally agreed better; Claude's weakly informative set was the only one that stayed informative (non-zero mean) and suitably wide. The message is that LLM knowledge of direction is already usable, but calibrating the width of an LLM-suggested prior is the unresolved problem.

What carries the argument

Two mechanisms carry the argument. The first is a structured elicitation prompt: the model is asked to state how it would use domain knowledge, propose at least two prior sets (moderate and weak), justify every normal prior's mean and standard deviation, and rate its own confidence in each set. This is what lets the paper attribute failures to calibration rather than to missing knowledge. The second is the evaluation metric, the Kullback-Leibler (KL) divergence from the Gaussian approximation of the maximum-likelihood estimator's distribution to each proposed prior; because the prior appears in the denominator of the integrand, a prior that is surprised by the data is penalized asymmetrically. Together these mechanisms separate 'does the model know the direction?' from 'does the model know the strength?'.

What would settle it

Run the same LLM elicitation on simulated datasets whose true regression coefficients are known and that contain no selection bias, then compute how often each suggested prior's central 50% interval covers the true coefficient across many replicates; the paper's overconfidence claim predicts that the moderately informative priors will cover far less than 50% of the time.

Watch

Extended reading notes

Core claim

The paper's central claim is that LLMs can act as knowledge-based prior elicitors for Bayesian regression only if the width of their suggested distributions is treated with suspicion. In both experiments every model assigned the correct sign to every coefficient, showing that the direction of an association can be reliably extracted from an LLM. However, the 'moderately informative' priors—which the models rated 60-65% confident—frequently placed most of their mass far from the maximum-likelihood estimate, producing high KL divergence; the weakly informative priors generally did better. By the KL-from-MLE benchmark, Claude's weakly informative priors ranked best on both datasets, with average divergences of 1.07 (heart disease) and 2.47 (concrete), while ChatGPT's zero-centered, extremely wide priors ranked worst. The paper concludes that the bottleneck is calibration: the models oscillate between overconfidence in their moderate priors and unnecessarily vague defaults in their weak priors, and that modest predictive gains over a frequentist model were not statistically significant.

Load-bearing premise

The load-bearing premise is that the one observed dataset used to compute the MLE distribution is a fair target for a prior, i.e., that the data are not severely biased; if the dataset is biased—for example, the heart-disease sample has a 46% disease prevalence—the KL rankings could be measuring data bias rather than prior quality.

Editorial extensions

If this is right

  • Practitioners can already use LLM suggestions to fix the sign or direction of coefficients in small-data or regularization settings, where the data alone cannot establish direction.
  • When an LLM rates a moderately informative prior as its most confident suggestion, that confidence should be discounted; the weakly informative set is the safer choice in the paper's experiments.
  • In large datasets, LLM priors should not be expected to improve predictive accuracy, because the likelihood dominates the posterior; any benefit is more likely to appear in small samples or out-of-distribution settings.
  • Prior-data conflict checks based on KL divergence to the MLE distribution can be added to a Bayesian workflow with only the observed data, without needing posterior samples.
  • Requiring the model to justify a non-zero expected effect prevents the 'unnecessarily vague' zero-mean default that hurt ChatGPT and Gemini's weakly informative priors.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Going beyond the paper: the same prompt could be run on simulated data with known coefficients, and the coverage of each suggested prior could be measured directly; this would separate general overconfidence from artifacts of the particular datasets used here.
  • Going beyond the paper: one cheap fix suggested by the results is a post-processing step that keeps the LLM's sign and mean but inflates the standard deviation until the prior predictive overlaps a validation split, borrowing the model's knowledge while forcing data-based scale.
  • Going beyond the paper: the finding that only Claude avoided the zero-mean default hints that prompt instructions alone do not guarantee calibration-friendly behavior; testing models with identical prompts across repeated runs would show whether this is a stable model difference or a sampling effect.
  • Going beyond the paper: because the heart-disease data are case-heavy, an unbiased replication would tell whether part of the prior-data gap is the dataset's selection bias rather than the LLM's miscalibration.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes using large language models (LLMs) to elicit informative prior distributions for Bayesian regression models. The authors design a multi-part prompt that asks an LLM to justify its choices, propose both moderately and weakly informative prior sets, and assign confidence scores. They evaluate Claude Opus, Gemini 2.5 Pro, and ChatGPT-4o-mini on two datasets: a Cleveland heart-disease logistic regression and a concrete compressive-strength linear regression. Prior quality is measured by the Kullback-Leibler divergence between a Gaussian approximation of the MLE sampling distribution and each suggested prior. The reported results are that all LLMs identify the correct direction for every association, that moderately informative priors are often overconfident, and that Claude's weakly informative priors achieve the lowest average KL divergence, making Claude and Gemini better than ChatGPT overall. Prediction experiments show only small, statistically insignificant improvements over frequentist baselines.

Significance. If the central claims hold, the paper offers a practical, low-cost route to prior elicitation and identifies a specific failure mode: LLMs are sign-calibrated but width-miscalibrated. The authors deserve credit for explicitly acknowledging the data-bias assumption behind their KL-to-MLE benchmark, for using a structured elicitation prompt with confidence scores, and for including prediction comparisons with a corrected paired t-test. The qualitative findings about direction recall and moderate-prior overconfidence appear well supported by the figures. However, the quantitative ranking of LLMs is currently based on single-run KL values with no uncertainty assessment, and the full prompts and code are not available in the manuscript. These issues make the headline ordering provisional rather than established.

major comments (4)
  1. [§2.2, Eqs. (2)–(3); Tables 2 and 5] The quantitative rankings rest on KL divergences computed once per LLM and prior set, with no assessment of sampling variability. There is no repeated LLM sampling, no bootstrap or Monte Carlo error for the Gaussian approximation of the MLE distribution, and no sensitivity analysis. For example, in Table 2 the difference between Claude weak (average KL 1.07) and Gemini weak (1.55) is reported as a ranking, but the reader cannot tell whether this gap exceeds the variability of the benchmark itself, and in Table 5 the analogous comparison is even tighter (Gemini moderate 1.09 versus Claude moderate 1.67). To make the central ordering 'Claude and Gemini provided better priors than ChatGPT' and 'Claude's weakly informative priors were the best' load-bearing, the authors should provide uncertainty intervals, repeated LLM draws, and ideally bootstrap resampling of the MLE distribution.
  2. [§2.2 and §4, Eq. (3)] The KL-to-MLE benchmark assumes that the MLE distribution computed from one observed dataset is a valid reference for prior quality, and the paper explicitly states in Section 4 that the analysis proceeds under the assumption that the data are not severely biased. This assumption is load-bearing: if the Cleveland data carry case-control oversampling (46% prevalence), the disagreement between population-level prior knowledge and the data could reflect data bias rather than poor prior calibration, and all KL-based rankings in Tables 2 and 5 would change. Because the paper itself raises this concern, the authors should add a concrete sensitivity analysis, for instance re-computing rankings with the intercept and sex coefficients excluded, using an alternative reference distribution (e.g., a posterior under a weakly informative prior), or applying prior-predictive checks. As written, the caveat is stated but not operationalized, leaving the main quantitative claim vulnerable.
  3. [§2.1, Listing 1, and 'A link to our code'] The exact prompt is a central methodological contribution, but only a skeleton is shown in Listing 1 (with material marked 'not included for brevity') and the code link is deferred to the published version. Without the full prompts, the temperature settings, the number of LLM runs, the date of API access, and the raw outputs, the results cannot be reproduced or assessed for sensitivity to prompt wording. Given that the paper's stated contribution includes 'an extensive prompt,' this material should be included in the supplement or as an appendix, not merely promised for a future version.
  4. [§2.2, Eq. (3) and Figures 2–5] The Kullback-Leibler divergence is used asymmetrically, with the MLE distribution in the numerator and the prior in the denominator. The text interprets this as penalizing priors that are surprised by the data, but the asymmetry also means that very wide priors will receive small KL values regardless of whether they encode useful information. This is not necessarily a flaw, but it directly affects the interpretation of 'unnecessarily vague' priors: ChatGPt's wide weak priors have high KL in Table 2, which seems at odds with the usual behavior of KL for wide distributions. The authors should clarify why the weakly informative ChatGPT priors, described as 'extremely wide,' nevertheless yield some of the largest KL values (e.g., 6.25 for chol in Table 2), since this bears on whether the metric captures overconfidence or something else.
minor comments (5)
  1. [Abstract and §3] The model is called 'ChatGPT-4o-mini' in the abstract and Section 3 but 'ChatGPT o4-mini' in Listing 1; reference [10] cites 'Gemini 1.5 Pro' while the experiments use Gemini 2.5 Pro. Please make the model names and citations consistent and include exact version/date information.
  2. [Listing 1] The placeholder '(not inclded for the sake of brevity)' appears twice; 'inclded' is a typo and, more importantly, the omitted text is needed for reproducibility. Replace the placeholders with the full prompt or an appendix reference.
  3. [§2.2] The term 'Kullback-Leibler' is capitalized in the section heading but not in the text; standardize the capitalization. Additionally, the definition in Eq. (3) writes 'DKL(p||pMLE)' while the text sometimes describes the reverse direction; please ensure notation is consistent with the actual direction computed.
  4. [Tables 2 and 5] The 'Avg Rank' row is reported without explaining how the ranking is computed across variables. Please provide the definition (e.g., rank within each variable across the six prior sets) so the reader can interpret the summary.
  5. [References] Reference [9] says '3rd ed edition' (duplicate 'edition') and reference [22] has a stray formatting artifact in the author name. These are minor typographical issues.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: LLM priors are elicited independently of the data and evaluated against an external KL-to-MLE benchmark.

full rationale

The paper's central derivation is not circular. The LLM-suggested priors are generated from a prompt that asks for domain knowledge and simulated literature review, with no access to the evaluation datasets; the suggested hyperparameters are then compared, via KL divergence in Eq. (3), to a Gaussian approximation of the MLE distribution estimated from the data. No fitted parameter is fed back into the prior elicitation, and no quantity that defines the result is estimated from the target outcome. The KL-to-MLE criterion is a normative benchmark choice, explicitly motivated in Section 2.2 and openly criticized within the paper in Section 4, where the authors state 'we have proceeded under the assumption that the data are not severely biased.' This acknowledged assumption makes the benchmark debatable as a statistical correctness question, but it does not make the derivation equivalent to its inputs. The only self-citation is to prior work [4] on LLM-based prior elicitation, and it is background context rather than a load-bearing justification for the paper's claims. The qualitative and quantitative claims about LLM prior quality rest on the elicited priors and the independent data-based benchmark, not on a self-referential chain.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters are fitted by the authors; the LLM-suggested hyperparameters are outputs, not inputs. The ledger contains three unproved premises: asymptotic normality of the MLE, the normative benchmark choice, and the absence of severe data bias.

assumptions (3)
  • standard math The maximum likelihood estimator is asymptotically Gaussian.
    Invoked in Section 2.2 to justify approximating the MLE distribution as Gaussian for the KL comparison; requires regularity conditions and adequate sample size.
  • ad hoc to paper Closeness of a prior to the MLE distribution is a valid measure of prior quality.
    The paper uses KL divergence from the MLE distribution as its quality metric and acknowledges in Section 4 that a prior is not inherently wrong simply because it diverges from observed data.
  • domain assumption The analyzed datasets are not severely biased.
    Explicitly assumed in Section 4: 'we have proceeded under the assumption that the data are not severely biased.' This is load-bearing because data bias would make the MLE benchmark misleading.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Using Large Language Models to Suggest Informative Prior Distributions in Bayesian Statistics." pith.science (2026). https://pith.science/paper/55COL3QN

@misc{pith2026250621964,
  author       = {Pith},
  title        = {Pith review of: Using Large Language Models to Suggest Informative Prior Distributions in Bayesian Statistics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/55COL3QN}},
  note         = {Machine review of arXiv:2506.21964}
}
read the original abstract

Selecting prior distributions in Bayesian statistics is challenging, resource-intensive, and subjective. We analyze using large-language models (LLMs) to suggest suitable, knowledge-based informative priors. We developed an extensive prompt asking LLMs not only to suggest priors but also to verify and reflect on their choices. We evaluated Claude Opus, Gemini 2.5 Pro, and ChatGPT-4o-mini on two real datasets: heart disease risk and concrete strength. All LLMs correctly identified the direction for all associations (e.g., that heart disease risk is higher for males). The quality of suggested priors was measured by their Kullback-Leibler divergence from the maximum likelihood estimator's distribution. The LLMs suggested both moderately and weakly informative priors. The moderate priors were often overconfident, resulting in distributions misaligned with the data. In our experiments, Claude and Gemini provided better priors than ChatGPT. For weakly informative priors, a key performance difference emerged: ChatGPT and Gemini defaulted to an "unnecessarily vague" mean of 0, while Claude did not, demonstrating a significant advantage. The ability of LLMs to identify correct associations shows their great potential as an efficient, objective method for developing informative priors. However, the primary challenge remains in calibrating the width of these priors to avoid over- and under-confidence.

Figures

Figures reproduced from arXiv: 2506.21964 by the authors.

Figure 1
Figure 1. Cement strength example: Visualization of prior, MLE, and posterior distributions. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Heart disease example: MLE distributions and the moderately informative LLM prior [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Heart disease example: MLE distributions and the weakly informative LLM prior [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Concrete strength example: MLE distributions and the moderately informative LLM [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Concrete strength example: MLE distributions and the weakly informative LLM prior [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

26 extracted references · 17 canonical work pages

  1. [1]

    LLMs in education: Novel perspectives, challenges, and opportunities

    Bashar Alhafni, Sowmya Vajjala, Stefano Bann` o, Kaushal Kumar Maurya, and Ekaterina Kochmar. LLMs in education: Novel perspectives, challenges, and opportunities. arXiv preprint arXiv:2409.11917, 2024

    arXiv 2024

  2. [2]

    Claude 3 Opus

    Anthropic. Claude 3 Opus. https://www.anthropic.com/claude, 2024. Accessed: 2024- 06-25

    work page 2024

  3. [3]

    Current applications and challenges in large language models for patient care: a systematic review

    Felix Busch, Lena Hoffmann, Christopher Rueger, Elon HC van Dijk, Rawen Kader, Es- teban Ortiz-Prado, Marcus R Makowski, Luca Saba, Martin Hadamitzky, Jakob Nikolas Kather, et al. Current applications and challenges in large language models for patient care: a systematic review. Communications Medicine, 5(1):26, 2025

    work page 2025

  4. [4]

    Using large language models for expert prior elicitation in predictive modelling

    Alexander Capstick, Rahul G Krishnan, and Payam Barnaghi. Using large language models for expert prior elicitation in predictive modelling. arXiv preprint arXiv:2411.17284, 2024

    arXiv 2024

  5. [5]

    Statistical inference

    George Casella and Roger Berger. Statistical inference. CRC press, 2024

    2024

  6. [6]

    Leveraging large language models in finance: Pathways to responsible adoption, 2024

    European Securities and Markets Authority (ESMA). Leveraging large language models in finance: Pathways to responsible adoption, 2024. Available at: https://www.esma. europa.eu/

    work page 2024

  7. [7]

    Checking for prior-data conflict

    Michael Evans and Hadas Moshonov. Checking for prior-data conflict. Bayesian Analysis, 1, 12 2006

    work page 2006

  8. [8]

    Bayesian concept bottleneck models with llm priors

    Jean Feng, Avni Kothari, Luke Zier, Chandan Singh, and Yan Shuo Tan. Bayesian concept bottleneck models with llm priors. arXiv preprint arXiv:2410.15555, 2024

    arXiv 2024

Show all 26 references
  1. [9]

    Carlin, Hal S

    Andrew Gelman, John B. Carlin, Hal S. Stern, David B. Dunson, Aki Vehtari, and Donald B. Rubin. Bayesian Data Analysis. CreateSpace, United States, 3rd ed edition, 2013

    2013

  2. [10]

    Gemini 1.5 Pro

    Google. Gemini 1.5 Pro. https://gemini.google.com/, 2024. Accessed: 2024-06-25

    2024

  3. [11]

    Automated prior elicitation from large language models for bayesian logistic regression

    Henry Gouk and Boyan Gao. Automated prior elicitation from large language models for bayesian logistic regression. In The 3rd International Conference on Automated Machine Learning, 2024

    2024

  4. [12]

    Accurate predictions on small data with a tabular foundation model

    Noah Hollmann, Samuel M¨ uller, Lennart Purucker, Arjun Krishnakumar, Max K¨ orfer, Shi Bin Hoo, Robin Tibor Schirrmeister, and Frank Hutter. Accurate predictions on small data with a tabular foundation model. Nature, 637(8045):319–326, 2025

    2025

  5. [13]

    Heart Disease

    Andras Janosi, William Steinbrunn, Matthias Pfisterer, and Robert Detrano. Heart Disease. UCI Machine Learning Repository, 1989. DOI: https://doi.org/10.24432/C52P4X

    1989 doi

  6. [14]

    Survey of hallucination in natural language generation

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM computing surveys, 55(12):1–38, 2023

    2023

  7. [15]

    The Fram- ingham Heart Study and the epidemiology of cardiovascular diseases: a historical perspec- tive

    Syed S Mahmood, Daniel Levy, Ramachandran S Vasan, and Thomas J Wang. The Fram- ingham Heart Study and the epidemiology of cardiovascular diseases: a historical perspec- tive. The Lancet, 383(9921):999–1008, 2014

    2014

  8. [16]

    Prior knowledge elicitation: The past, present, and future

    Petrus Mikkola, Osvaldo A Martin, Suyog Chandramouli, Marcelo Hartmann, Oriol Abril Pla, Owen Thomas, Henri Pesonen, Jukka Corander, Aki Vehtari, Samuel Kaski, et al. Prior knowledge elicitation: The past, present, and future. Bayesian Analysis, 19(4):1129–1161, 2024. 16

    2024

  9. [17]

    Inference for the generalization error

    Claude Nadeau and Yoshua Bengio. Inference for the generalization error. Advances in neural information processing systems, 12, 1999

    1999

  10. [18]

    A survey of large language models for financial applications: Progress, prospects and challenges

    Yuqi Nie, Yaxuan Kong, Xiaowen Dong, John M Mulvey, H Vincent Poor, Qingsong Wen, and Stefan Zohren. A survey of large language models for financial applications: Progress, prospects and challenges. arXiv preprint arXiv:2406.11903, 2024

    2024 arXiv

  11. [19]

    Checking for prior-data conflict using prior-to-posterior divergences

    David J Nott, Xueou Wang, Michael Evans, and Berthold-Georg Englert. Checking for prior-data conflict using prior-to-posterior divergences. Statistical Science, 35(2):234–253, 2020

    2020

  12. [20]

    ChatGPT-4o-mini

    OpenAI. ChatGPT-4o-mini. https://openai.com/chatgpt, 2024. Accessed: 2024-06-25

    2024

  13. [21]

    Llm processes: Numerical predictive distributions conditioned on natural language

    James Requeima, John Bronskill, Dami Choi, Richard Turner, and David K Duvenaud. Llm processes: Numerical predictive distributions conditioned on natural language. Advances in Neural Information Processing Systems, 37:109609–109671, 2024

    2024

  14. [22]

    Approximate bayesian inference for latent gaussian models by using integrated nested laplace approximations

    H ˚ avard Rue, Sara Martino, and Nicolas Chopin. Approximate bayesian inference for latent gaussian models by using integrated nested laplace approximations. Journal of the Royal Statistical Society Series B: Statistical Methodology, 71(2):319–392, 2009

    2009

  15. [23]

    MONICA: Monograph and Multimedia Sourcebook: World’s largest study of heart disease, stroke, risk factors, and population trends, 1979-2002

    Hugh Tunstall-Pedoe, editor. MONICA: Monograph and Multimedia Sourcebook: World’s largest study of heart disease, stroke, risk factors, and population trends, 1979-2002. World Health Organization, Geneva, 2003. Prepared on behalf of the WHO MONICA Project

    1979

  16. [24]

    Large language models for education: A survey and outlook

    Shen Wang, Tianlong Xu, Hang Li, Chaoli Zhang, Joleen Liang, Jiliang Tang, Philip S Yu, and Qingsong Wen. Large language models for education: A survey and outlook. arXiv preprint arXiv:2403.18105, 2024

    2024 arXiv

  17. [25]

    Concrete Compressive Strength

    I-Cheng Yeh. Concrete Compressive Strength. UCI Machine Learning Repository, 1998. DOI: https://doi.org/10.24432/C5PK67

    1998 doi

  18. [26]

    Revolutionizing health care: The transforma- tive impact of large language models in medicine

    Kuo Zhang, Xiangbin Meng, Xiangyu Yan, Jiaming Ji, Jingqian Liu, Hua Xu, Heng Zhang, Da Liu, Jingjia Wang, Xuliang Wang, et al. Revolutionizing health care: The transforma- tive impact of large language models in medicine. Journal of Medical Internet Research, 27:e59069, 2025. 17

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.