Pith. sign in

REVIEW 3 major objections 6 minor 25 references

Testing Uncertainty of Large Language Models for Physics Knowledge and Reasoning

T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Error rate and response entropy are coupled on physics multiple-choice questions, but the coupling fails exactly where reasoning is required.

desk verdict Descriptive, honest workshop paper whose headline bell curve is derived in its own appendix; the real empirical content is the unquantified scatter, which needs statistical support before the category claim holds. read the letter →

arxiv 2411.14465 v1 pith:2USTBN6G submitted 2024-11-18 cs.CL cs.LG

Elizaveta Reganova , Peter Steinbach This is my paper · ORCID
classification cs.CLcs.LG
keywords largelanguagemodelsuncertaintyestimationhallucinationphysicsmultiplechoiceentropyaccuracy-certaintytrade-offreasoningvsknowledgeretrievalfew-shotprompting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish how reliably a large language model's own response consistency signals correctness on physics questions. Sampling each of 823 multiple-choice questions 20 times from four models, the authors measure answer variability as entropy and compare it with error rate. They find that low uncertainty usually accompanies correct answers, but not always: a noticeable set of questions receive wrong answers with high certainty, and this mismatch grows as questions move from recalling definitions and facts to single- and multi-step reasoning. The overall error-rate-versus-entropy scatter takes a broad horizontal bell-shaped form, which the paper shows in an appendix to be largely a mathematical consequence of how the two quantities are constructed from the same response counts.

What carries the argument

The carrying object is the pairwise plot of error rate (1 minus accuracy) against response entropy $H(Y|x,h)=-\sum_i p(y_i|x,h)\ln p(y_i|x,h)$, computed from a question's 20 repeated answers, together with the parameterized family of theoretical curves derived in Appendix E. For a question with only two distinct answers, one correct and one incorrect, entropy is forced to $-(1-e)\ln(1-e)-e\ln e$ where $e$ is the error rate; with three, four, or five distinct answers the same relation holds as a family of curves with one or more free probabilities. This identity explains the bell-shaped envelope of the histograms and shifts the empirical content to the density of questions across the envelope.

What would settle it

A reader could resample each question with N=200 at the same temperature and rebuild the error-rate-versus-entropy histograms; if the high-certainty, high-error points in the S and M categories disappear or shift substantially, the reported asymmetry is an artifact of the 20-sample estimate. Alternatively, permuting the correct-answer labels while keeping response counts fixed would reproduce the same envelope, confirming that curve shape alone cannot identify hallucination.

Watch

Extended reading notes

Core claim

The central claim is that accuracy and uncertainty are coupled for LLMs on physics multiple-choice questions, but the coupling is not uniform: most models provide accurate replies when they are certain, but this is by no means a general behavior, and the asymmetry between accuracy and uncertainty intensifies as questions demand more logical reasoning while staying comparatively sharp for knowledge retrieval. Concretely, single-step (S) and multi-step (M) reasoning questions show more high-certainty wrong answers and more high-diversity scattered responses across models, whereas definition (D) and fact (F) categories concentrate accurate, low-entropy replies. The authors present this through two-dimensional histograms of error rate versus entropy for four models and five question categories, interpreting the high-certainty, high-error region as a signature of hallucination.

Load-bearing premise

The analysis assumes that asking each question 20 times at temperature 0.7 gives a stable estimate of a model's response distribution, and that the bell-shaped curve's shape is an empirical finding rather than a consequence of how error rate and entropy are computed from the same 20 answers.

Editorial extensions

If this is right

  • In knowledge-retrieval categories (D and F), low entropy is a fairly reliable indicator of a correct answer for the tested models.
  • In reasoning categories (S and M), confidence-like signals from sampling degrade; users should not read low entropy as correctness in multi-step physics problems.
  • The bell-shaped error-rate-versus-entropy envelope is a mathematical consequence of the measurement, so comparisons between models must be made through scatter density and category-wise patterns, not through the curve's gross shape.
  • Larger models in this set (Mixtral) produce lower diversity, while GPT-3.5-turbo shows higher diversity, implying consistency differences track model choice and prompting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: if the 20-sample entropy proves unstable, the reported scatter could change with sample size; re-running with N=100 or N=500 would test whether high-certainty errors are robust or a small-sample artifact.
  • Beyond the paper: the same entropy-versus-error-rate identity applies to any discrete classifier with repeated sampling, so the method transfers to other domains and could be used as a cheap hallucination screen for non-physics MCQ benchmarks.
  • Beyond the paper: the paper's category ordering D < F < C < S < M suggests a testable gradient: if prompting style shifts (e.g., chain-of-thought) reduce the asymmetry in S and M, the reported gap is partly a prompting effect rather than an intrinsic reasoning deficit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper evaluates four LLMs (Llama3.1-8B-Instruct, Mixtral-8x7B-Instruct-v0.1, Mistral-7B-Instruct-v0.3, GPT-3.5-turbo) on the mlphys101 physics multiple-choice dataset, which contains 823 questions in five categories (D, F, C, S, M). For each question, the authors sample N=20 responses at temperature 0.7, compute the response entropy and the error rate, and display 2D histograms of error rate versus entropy overall and per category. The headline claims are that the error-rate/entropy relationship forms a broad bell-shaped distribution, that most models are accurate when certain but not universally so, and that the accuracy–uncertainty asymmetry intensifies as questions demand more logical reasoning (categories S and M). The paper also makes a methodological point, derived in Appendix E, that the bell-shaped envelope corresponds to a family of theoretical curves parameterized by the number of distinct incorrect responses.

Significance. If the empirical claims were properly supported, the paper would offer a useful contribution to evaluating LLM trustworthiness in a scientific domain: the mlphys101 dataset is a novel resource, and the paper ships code and data, which is commendable. However, the central descriptive claims are weakened by a mathematical coupling between the two plotted quantities and by the absence of any statistical quantification of the category differences. The dataset and the general framework have value, but the analysis needs substantial strengthening before the reported conclusions can be accepted.

major comments (3)
  1. [Section 3 and Appendix E, Eqs. (1)–(4)] The abstract and Section 3 present the 'broad horizontal bell-shaped distribution' as an empirical discovery, but Appendix E shows that the envelope is a mathematical identity: for any question where the model produces exactly two distinct responses, one correct and one incorrect, entropy is exactly the binary entropy of the error rate (Eq. 1). The 2D histograms must therefore lie within this deterministic family of curves, so the bell shape carries no empirical information. The only empirical content is the density of points within the envelope. Please reframe the claims to focus on the scatter and its category dependence, and quantify deviations from the theoretical curves rather than presenting the envelope itself as a finding.
  2. [Section 3, Figure 3 (Appendix C)] The claim that the accuracy–uncertainty asymmetry 'intensifies' for single-step (S) and multi-step (M) reasoning questions is read off 2D histograms without any statistical test, error bar, or conditional analysis. Because the categories differ substantially in size (D:153, F:138, C:238, S:223, M:71) and in marginal accuracy, a visually higher density of high-certainty error points in S/M could simply reflect lower overall accuracy or larger category counts. The paper should report, for example, the conditional error rate given entropy intervals per category with confidence intervals, or at least provide counts of high-certainty errors and a formal test (e.g., Fisher's exact test or a logistic regression with category as a predictor) to support the asymmetry claim.
  3. [Sections 2.2 and 2.3] The reliability of the scatter plots rests on the stability of entropy and accuracy estimates from N=20 responses per question, but the paper gives no error bars on either quantity and does not report how many responses were excluded as malformed (Section 2.2 says such replies were 'replaced with None and excluded'). Since entropy and error rate are computed from the same 20 counts, sampling noise can generate apparent scatter even for a fixed underlying distribution. Please report the number of excluded responses per model, the effective number of responses used per question, and bootstrap or analytical confidence intervals for the plotted quantities.
minor comments (6)
  1. [Section 2.2] Typo: 'infracture' should be 'infrastructure'.
  2. [Section 3] 'We can hypothesis at this point' should be 'We can hypothesize at this point'.
  3. [References] Several references are incomplete: reference [8] has no year and reference [9] has a missing year. Please check all entries for completeness.
  4. [Appendix B] The few-shot prompt example is hard to read because the code block contains irregular spacing. Format it as a clean code block for clarity.
  5. [Section 2.1 and Abstract] The dataset is described as 'university-level' in Section 2.1 but as 'high-school grade physics questions' in the abstract; clarify which level applies and keep the description consistent.
  6. [Introduction] The claim that this is 'the first publication which focuses on the trustworthiness of LLM answers in physics reasoning' is strong; consider softening the novelty claim or providing a more systematic comparison with prior works on uncertainty estimation in QA.

Circularity Check

1 steps flagged · score 6.0 of 10

The headline bell-shaped error-vs-entropy curve is a mathematical identity of the two metrics (Appendix E, Eq. 1), not an independent empirical finding; the category-asymmetry claim remains empirical but is not quantified.

  1. self definitional [Section 3 (Figure 2) and Appendix E (Eq. 1)]
    "The 2D histograms of ”1-Accuracy” (or Error Rate) versus Entropy for all models exhibit a similar bell-shaped distribution (for a detailed mathematical explanation of the curve’s shape, refer to Appendix E). ... H(Y |x, h) =−(1 − error_rate) ∗ ln(1 − error_rate) − error_rate ∗ ln(error_rate). (1)"

    Both plotted axes are computed from the same N=20 per-question letter counts (Section 2.3: p(y_i|x,h)=count(y_i)/N, H is the Shannon entropy of those same counts). Therefore the possible points are not free data: for a two-response question, Eq. (1) fixes entropy as the binary entropy of error rate, and for three or more response types Eqs. (2)-(4) give parameterized families, so the broad bell-shaped envelope is a theorem about the plotting variables. Reporting that the relationship 'exposes a broad horizontal bell-shaped distribution' as a finding is thus presenting a definitional identity as an empirical observation; the only genuinely empirical content is the density/scatter of questions within the forced envelope and its category-level variation.

full rationale

The paper's secondary descriptive finding about the curve shape reduces by construction, and the abstract packages it as one of 'Our findings'. However, the main behavioral claims—that models are usually accurate when certain but not always, and that the accuracy/certainty asymmetry differs between knowledge-retrieval (D/F) and reasoning (S/M) categories—are empirical statements about where questions fall inside that mathematical envelope. They are not forced by the metrics, though they are presented only as unquantified 2D histograms without error bars or a conditional accuracy-given-entropy analysis; that is a statistical-support weakness, not circularity. The mlphys101 dataset is self-cited and unpublished, but as data it is not a theorem that makes the conclusions circular. Overall, one central expository finding is definitionally forced, so the paper is partially circular but not wholly so.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central analysis rests on the mlphys101 ground truth, on the assumption that 20 samples at temperature 0.7 approximate each model's predictive distribution, and on the standard entropy formula. No new entities are introduced. The theoretical curve in Appendix E is a derivation, not a fitted model.

free parameters (2)
  • temperature = 0.7
    Set by hand for all models; directly controls the randomness of sampled answers and therefore the entropy estimates, with no sensitivity analysis.
  • number of repetitions N = 20
    Chosen by hand; the entropy and accuracy for each question are estimated from only 20 samples, with no reported confidence intervals.
assumptions (3)
  • domain assumption The mlphys101 dataset's labeled correct answers are accurate
    All accuracy calculations depend on these labels being correct; the dataset is from a companion, not-yet-published paper.
  • domain assumption 20 repeated samples at temperature 0.7 approximate the model's predictive distribution
    Entropy and accuracy are computed from these samples; the paper provides no statistical validation that 20 is sufficient.
  • standard math Output diversity (entropy) is a valid proxy for LLM uncertainty
    Standard in uncertainty estimation, but it measures only the spread of letters, not semantic equivalence of different wrong answers.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Testing Uncertainty of Large Language Models for Physics Knowledge and Reasoning." pith.science (2026). https://pith.science/paper/2USTBN6G

@misc{pith2026241114465,
  author       = {Pith},
  title        = {Pith review of: Testing Uncertainty of Large Language Models for Physics Knowledge and Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2USTBN6G}},
  note         = {Machine review of arXiv:2411.14465}
}
read the original abstract

Large Language Models (LLMs) have gained significant popularity in recent years for their ability to answer questions in various fields. However, these models have a tendency to "hallucinate" their responses, making it challenging to evaluate their performance. A major challenge is determining how to assess the certainty of a model's predictions and how it correlates with accuracy. In this work, we introduce an analysis for evaluating the performance of popular open-source LLMs, as well as gpt-3.5 Turbo, on multiple choice physics questionnaires. We focus on the relationship between answer accuracy and variability in topics related to physics. Our findings suggest that most models provide accurate replies in cases where they are certain, but this is by far not a general behavior. The relationship between accuracy and uncertainty exposes a broad horizontal bell-shaped distribution. We report how the asymmetry between accuracy and uncertainty intensifies as the questions demand more logical reasoning of the LLM agent, while the same relationship remains sharp for knowledge retrieval tasks.

Figures

Figures reproduced from arXiv: 2411.14465 by the authors.

Figure 1
Figure 1. Entropy obtained from the distribution of answers to single questions of the [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Two-dimensional Histogram of Error Rate (1 - Accuracy) vs. Entropy across Models. The [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Accuracy-certainty trade-off for each LLM in five question categories [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Two-dimensional Histogram of Error Rate (1 - Accuracy) vs. Entropy across Models with [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: A. Two-dimensional histogram of (1 - Accuracy) vs. Entropy for the Mistral 7B model, [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 10 canonical work pages

  1. [1]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Flo- rencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Bernadett-Shapiro, Christopher Bern...

    arXiv 2024

  2. [2]

    Chatgpt for good? on opportunities and challenges of large language models for education

    Enkelejda Kasneci, Kathrin Seßler, Stefan Küchemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan Günnemann, Eyke Hüllermeier, et al. Chatgpt for good? on opportunities and challenges of large language models for education. Learning and individual differences, 103:102274, 2023

    work page 2023

  3. [3]

    Toolqa: A dataset for llm question answering with external tools, 2023

    Yuchen Zhuang, Yue Yu, Kuan Wang, Haotian Sun, and Chao Zhang. Toolqa: A dataset for llm question answering with external tools, 2023. URL https://arxiv.org/abs/2306.13304

    arXiv 2023

  4. [4]

    Trustworthy llms: a survey and 6 guideline for evaluating large language models’ alignment, 2024

    Yang Liu, Yuanshun Yao, Jean-Francois Ton, Xiaoying Zhang, Ruocheng Guo, Hao Cheng, Yegor Klochkov, Muhammad Faaiz Taufiq, and Hang Li. Trustworthy llms: a survey and 6 guideline for evaluating large language models’ alignment, 2024. URL https://arxiv.org/ abs/2308.05374

    arXiv 2024

  5. [5]

    Detecting hallucinations in large language models using semantic entropy

    Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. Detecting hallucinations in large language models using semantic entropy. Nature, 630(8017):625–630, 2024

    2024

  6. [6]

    Lm-polygraph: Uncertainty estimation for language models,

    Ekaterina Fadeeva, Roman Vashurin, Akim Tsvigun, Artem Vazhentsev, Sergey Petrakov, Kirill Fedyanin, Daniil Vasilev, Elizaveta Goncharova, Alexander Panchenko, Maxim Panov, Timothy Baldwin, and Artem Shelmanov. Lm-polygraph: Uncertainty estimation for language models,

    work page

  7. [7]

    Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods

    Eyke Hüllermeier and Willem Waegeman. Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods. Machine Learning, 110(3):457–506, March

    work page

  8. [8]

    A survey of uncertainty in deep neural networks

    Jakob Gawlikowski, Cedrique Rovile Njieutcheu Tassi, Mohsin Ali, Jongseok Lee, Matthias Humt, Jianxiang Feng, Anna Kruspe, Rudolph Triebel, Peter Jung, Ribana Roscher, Muhammad Shahzad, Wen Yang, Richard Bamler, and Xiao Xiang Zhu. A survey of uncertainty in deep neural networks. URL http://arxiv.org/abs/2107.03342

    arXiv

Show all 25 references
  1. [9]

    Rajendra Acharya, Vladimir Makarenkov, and Saeid Nahavandi

    Moloud Abdar, Farhad Pourpanah, Sadiq Hussain, Dana Rezazadegan, Li Liu, Mohammad Ghavamzadeh, Paul Fieguth, Xiaochun Cao, Abbas Khosravi, U. Rajendra Acharya, Vladimir Makarenkov, and Saeid Nahavandi. A review of uncertainty quantification in deep learning: Techniques, applic...

  2. [10]

    Can multiple- choice questions really be useful in detecting the abilities of llms?, 2024

    Wangyue Li, Liangzhi Li, Tong Xiang, Xiao Liu, Wei Deng, and Noa Garcia. Can multiple- choice questions really be useful in detecting the abilities of llms?, 2024. URL https:// arxiv.org/abs/2403.17752

    2024 arXiv

  3. [11]

    Uncertainty in deep learning

    Yarin Gal et al. Uncertainty in deep learning. 2016

    2016

  4. [12]

    Buczek, C

    Marcel Völschow, P. Buczek, C. Carreno-Mosquera, E. Reganova, J. Roldan-Rodriguez, P. Stein- bach, and A. Strube. mlphys101 - exploring the performance of large-language models in multilingual undergraduate physics education. publication submitted, but unpublished, 2024. URL h...

    2024

  5. [13]

    Piqa: Reasoning about physical commonsense in natural language, 2019

    Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. Piqa: Reasoning about physical commonsense in natural language, 2019. URL https://arxiv.org/abs/ 1911.11641

    2019 arXiv

  6. [14]

    Yi Wang, Jiafei Duan, Dieter Fox, and Siddhartha Srinivasa. NEWTON: Are large language models capable of physical reasoning? In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023 , pages 9743–9758, Singapor...

    2023 doi

  7. [15]

    Prompting

    Meta. Prompting. https://llama.meta.com/docs/how-to-guides/prompting/ , 2024. Accessed: 2024-09-10

    2024

  8. [16]

    Alice in wonderland: Simple tasks showing complete reasoning breakdown in state-of-the-art large language models,

    Marianna Nezhurina, Lucia Cipolina-Kun, Mehdi Cherti, and Jenia Jitsev. Alice in wonderland: Simple tasks showing complete reasoning breakdown in state-of-the-art large language models,

  9. [17]

    Are emergent abilities in large language models just in-context learning?, 2024

    Sheng Lu, Irina Bigoulaeva, Rachneet Sachdeva, Harish Tayyar Madabushi, and Iryna Gurevych. Are emergent abilities in large language models just in-context learning?, 2024. URL https: //arxiv.org/abs/2309.01809

    2024 arXiv

  10. [18]

    Llm in-context recall is prompt dependent, 2024

    Daniel Machlab and Rick Battle. Llm in-context recall is prompt dependent, 2024. URL https://arxiv.org/abs/2404.08865. 7 A Appendix A.1 Dataset Examples As discussed in section 2.1, our dataset consists of 5 classes of questions. We provide one example for each here to provide...

    2024 arXiv

  11. [22]

    The model generates only correct responses for a given question

  12. [23]

    The model generates only incorrect responses

  13. [24]

    The model generates two distinct responses, one of which is correct and the other incorrect

  14. [25]

    When the model consistently produces correct responses, both the entropy and the error rate are zero (accuracy equals 1)

    The model generates three or more distinct responses, with at least one being correct. When the model consistently produces correct responses, both the entropy and the error rate are zero (accuracy equals 1). On the curve, this scenario corresponds to the bottom-left corner. A...

  15. [2021]

    doi: 10.1007/s10994-021-05946-3

    ISSN 1573-0565. doi: 10.1007/s10994-021-05946-3. URL http://dx.doi.org/10. 1007/s10994-021-05946-3

    doi

  16. [2023]

    URL https://arxiv.org/abs/2311.07383

    arXiv

  17. [2024]

    URL https://arxiv.org/abs/2406.02061

    arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.