Pith. sign in

REVIEW 3 major objections 6 minor 68 references

From General Reasoning to Domain Expertise: Uncovering the Limits of Generalization in Large Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that in six large language models, performance on a simulated bar exam is statistically uncorrelated with performance on classical reasoning tests, implying that narrow and broad cognitive abilities are disconnected in…

desk verdict The paper's headline claim of no transfer between legal reasoning and general rationality is an artifact of an underpowered n=7 correlation analysis, and the paper ships none of the data needed to check it. read the letter →

arxiv 2506.21580 v1 pith:BJS3LBDL submitted 2025-06-16 cs.CL cs.AIcs.CY

Dana Alsagheer , Yang Lu , Abdulrahman Kamal , Omar Kamal , Mohammad Kamal , Nada Mansour , Cosmo Yang Wu , Rambiba Karanjai
show 2 more authors
Sen Li Weidong Shi
This is my paper · ORCID
classification cs.CLcs.AIcs.CY
keywords LLMreasoningdomaingeneralizationlegalrationalitycognitivebiascorrelationanalysisMultistateBarExaminationWasonselectiontask
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the general reasoning ability of large language models is disconnected from their performance on specialized, knowledge-heavy tasks. Concretely, it measures six LLMs on 1,000 simulated Multistate Bar Examination questions and on 240 items from classical rationality tests, then correlates the two average scores for each model. In every model the correlation is weak and not statistically significant, and the authors conclude that domain expertise does not transfer to general reasoning. The stakes: if true, strong benchmark performance in law or medicine cannot be taken as evidence that a model reasons soundly.

What carries the argument

The load-bearing object is the per-model Pearson correlation coefficient between a narrow-domain score and a broad-reasoning score. The narrow score comes from 1,000 simulated Multistate Bar Examination questions across seven legal subjects; the broad score comes from 240 items spanning the Wason Selection Task, the Conjunction Fallacy Task, and two base-rate-neglect tasks. This single statistic carries the entire argument: the conclusion that legal expertise and general reasoning are unrelated rests on these six correlation coefficients and their p-values.

What would settle it

A concrete check would be to recompute the correlation using the underlying per-domain or per-item scores instead of one average law score and one average rationality score per model; if any model shows a statistically significant positive correlation across legal subdomains and rationality tasks, the disconnection claim collapses. A second check is to rerun identical prompts with temperature fixed: if the same model gives the same answer across repeated trials, the reported test-retest inconsistency is overturned.

Watch

Extended reading notes

Core claim

The paper's central claim is that narrow and broad cognitive abilities are disconnected in the intelligence structure of LLMs. For each of six models, the authors computed a Pearson correlation between average legal-reasoning accuracy (bar-exam questions across seven legal subjects) and average rationality accuracy (Wason Selection Task, Conjunction Fallacy Task, and two base-rate-neglect tasks). All six correlations were weak and statistically insignificant, with coefficients from -0.11 to 0.29 and p-values above 0.05. The authors read this as evidence that LLM performance is fragmented: expertise in one domain reflects pattern-based mimicry rather than an integrated reasoning capacity that generalizes across tasks.

Load-bearing premise

The conclusion depends on the unstated assumption that the Pearson correlations in Table 4 were computed over enough paired observations, with independent items, to detect a real relationship; the paper reports only two aggregate averages per model and p-values, so the sample size and unit of analysis cannot be verified.

Editorial extensions

If this is right

  • High scores on specialized benchmarks such as the simulated MBE cannot be used as evidence of general reasoning competence.
  • Increasing model size improves average performance in both legal and rationality tasks, but does not create a meaningful correlation between the two, so scaling alone does not yield transferable reasoning.
  • Fine-tuning or optimizing LLMs for a narrow domain is unlikely to improve broader rational decision-making.
  • Reliability is a separate axis: models can answer the same prompt differently across repetitions, so accuracy and consistency must be evaluated separately.
  • Auditing frameworks for high-stakes LLM deployment should include reasoning-stability and cognitive-bias checks, not just benchmark accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: correlating only six model-level averages makes a null result almost unavoidable; a stronger test would correlate many task-level scores within each model (legal subdomains versus individual rationality tasks) to see whether any shared reasoning factor appears.
  • Beyond the paper: if this disconnection holds, specialized leaderboards in law, medicine, or finance should not be treated as evidence of safe autonomous decision-making; deployment audits should add consistency and bias probes alongside accuracy.
  • Beyond the paper: the reported test-retest inconsistency can be quantified by repeatedly prompting the same model with temperature fixed and measuring answer entropy; the paper's qualitative finding predicts substantial entropy even on single-answer tasks.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper evaluates six LLMs (ChatGPT-4, Gemini 1.0, ChatGPT-3.5, Claude 2, Llama 3, Mistral) on a simulated Multistate Bar Examination (1,000 questions across seven legal domains) and on four classical rationality tasks (240 items: Wason Selection Task, Conjunction Fallacy, Stereotype and Diagnostic Base Rate Neglect). It computes per-model Pearson correlations between average legal scores and average rationality scores, reports no statistically significant correlations, and interprets this as evidence that narrow domain expertise and broad reasoning abilities are disconnected in LLMs. It also reports qualitative findings on cognitive biases and test-retest inconsistency, and proposes a human-centered auditing framework.

Significance. If the transfer claim were rigorously established, the paper would make a useful contribution to LLM evaluation: it would caution against interpreting high scores on domain benchmarks as evidence of general reasoning, and it would support the need for multi-faceted auditing protocols. The work has strengths: it uses external, publicly motivated benchmarks rather than self-defined tasks, the task battery (MBE plus classical rationality instruments) is sensible, and the paper explicitly offers datasets and scripts as supplementary material. However, the current statistical analysis is not adequate to support the central claim, and the qualitative 'inconsistency' evidence is undocumented.

major comments (3)
  1. [Section 4.1, Table 4] The reported correlation analysis is not reproducible because the sample size and unit of analysis are never stated. The p-values in Table 4 (e.g., r=0.14, p=0.76; r=0.29, p=0.53; r=−0.11, p=0.82) are exactly those expected with n=7 paired observations (df=5), and with n=7 the critical |r| for p<0.05 is approximately 0.754. The observed correlations of −0.11 to 0.29 therefore carry very wide confidence intervals that include substantial positive or negative associations, so the absence of significant p-values cannot support the claim that narrow and broad abilities are disconnected. The paper reports no equivalence test, no minimum detectable effect, and no multilevel or item-level analysis of the 1,000 legal items and 240 rationality items; as it stands, the central conclusion of Section 4.1 is unsupported by the evidence presented.
  2. [Section 4.3] The claim that models 'frequently returned different answers' to identical prompts under deterministic conditions is made without any supporting data. The manuscript does not report the number of repeated questions, the number of models for which inconsistency was observed, any measure of agreement (e.g., percent repeat consistency), or the API parameters used. Because this test-retest instability is described as 'most concerning' and is repeated in the conclusion, it needs either a quantitative summary or removal from the paper.
  3. [Methodology, Correlation Analysis] The operational definition of the paired observations used for the Pearson correlation is missing. The text says coefficients were calculated for each model, but Table 4 reports only two aggregate averages per model; it is unclear whether the correlation is computed over the seven legal domains, over the five evaluation rounds, or over some other grouping. Without this information, the statistical results cannot be checked, and the statement that the analysis is reproducible (Section 4) is not credible. The authors should specify the unit of analysis, state N, and ideally report item-level or multilevel estimates that pool information across the 1,240 available items.
minor comments (6)
  1. [Methodology and Table 1] There is a factual conflict in model sizes: the methodology text says ChatGPT-4 has '175 billion parameters,' while Table 1 lists ChatGPT-4 as 'Estimated 1-1.76T,' and the GPT-4 technical report [52] does not disclose a parameter count. Please correct and cite a source for the reported size.
  2. [Table 4] The header says 'Ranked by Model Size (Largest to Smallest),' but the rows do not follow the sizes given in Table 1; for example, ChatGPT-3.5 (≈175B) is placed after Gemini (≈70B) and before Claude (≈52–100B), and Claude is placed after ChatGPT-3.5. Either reorder the rows or change the header.
  3. [Section 4.2] The correlation coefficient is written as 'A' in the text (e.g., 'A = 0.14'), which appears to be a typo for 'r'; use a consistent symbol matching Table 4.
  4. [Section 4.1 and 4.3] Some symbols are missing in the text: 'All ?-values exceeded the 0.05 threshold' should read 'All p-values,' and 'con￿ated' should be 'conflated.'
  5. [References] Reference [7] is incomplete ('Unknown Author, Unknown Journal, 2023'), and references [62] and [63] are duplicate citations of the same Tversky and Kahneman 1974 article. Please complete or remove [7] and merge the duplicates.
  6. [Figures] Figure 2 is referenced but the caption alone is not sufficient; please ensure the figure appears in the final version with labeled axes and data points for each model. Figure 3 is also referenced but not visible in the manuscript text.

Circularity Check

0 steps flagged · score 1.0 of 10

No material circularity: the transfer conclusion is an empirical null result on external benchmarks; only a minor, non-load-bearing self-citation is present.

full rationale

The paper's central claim—that legal-domain performance and classical rationality-task performance are statistically uncorrelated—is an empirical result computed with Pearson's r on externally defined benchmarks (MBE legal domains and Wason/Conjunction/Base-Rate rationality tasks). No parameter is fitted to the transfer conclusion, no equation defines the conclusion into the inputs, and the analysis is self-contained: tasks, prompts, and scoring are independent of the paper's hypothesis. The only author-overlapping citation is reference [5], used in the related-work section to support the background statement that LLMs 'fail to generalize these skills to unfamiliar domains'; this is a minor self-citation and is not load-bearing for the new correlation analysis, which stands on its own measurements. The severe statistical-power limitation implied by the n=7 correlations in Table 4 (p-values matching df=5) is a validity concern about whether the null result supports the 'disconnected' conclusion, but it is not circularity: a low-power test does not make the conclusion identical to its inputs. The result is not forced by self-citation or by definition.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters or invented entities appear in the paper. The empirical claim rests on domain assumptions about what the tasks measure, an assumption of stability across averaged runs, and an implicit statistical model for the correlations that the paper never specifies.

assumptions (4)
  • domain assumption MBE performance is a valid measure of narrow domain-specific legal reasoning ability.
    Sections 2 and 4.1 treat the average law score on 1,000 MBE questions as the narrow-ability measure, without evidence that MBE accuracy isolates reasoning rather than memorization or test-taking skill.
  • domain assumption Wason Selection, Conjunction Fallacy, and base-rate neglect tasks measure domain-general rationality.
    Data Collection and Table 3 define these as rationality tasks, but the paper assumes they are broad measures rather than additional narrow skills with their own distributional quirks.
  • domain assumption Five averaged rounds per task yield stable model ability estimates.
    The Data Collection section says five rounds were averaged for reliability, but Section 4.3 later claims test-retest inconsistency under identical conditions, which undermines the stability assumption.
  • domain assumption Pearson correlation between aggregate scores can detect cross-domain transfer.
    Section 4.1 uses per-model correlations but never specifies the unit of analysis or sample size, so the statistical model is assumed rather than demonstrated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From General Reasoning to Domain Expertise: Uncovering the Limits of Generalization in Large Language Models." pith.science (2026). https://pith.science/paper/BJS3LBDL

@misc{pith2026250621580,
  author       = {Pith},
  title        = {Pith review of: From General Reasoning to Domain Expertise: Uncovering the Limits of Generalization in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BJS3LBDL}},
  note         = {Machine review of arXiv:2506.21580}
}
read the original abstract

Recent advancements in Large Language Models (LLMs) have demonstrated remarkable capabilities in various domains. However, effective decision-making relies heavily on strong reasoning abilities. Reasoning is the foundation for decision-making, providing the analytical and logical framework to make sound choices. Reasoning involves analyzing information, drawing inferences, and reaching conclusions based on logic or evidence. Decision-making builds on this foundation by applying the insights from reasoning to select the best course of action among alternatives. Together, these processes create a continuous cycle of thought and action aimed at achieving goals effectively. As AI technology evolves, there is a growing trend to train LLMs to excel in general reasoning. This study explores how the general reasoning capabilities of LLMs connect to their performance in domain-specific reasoning tasks.

Figures

Figures reproduced from arXiv: 2506.21580 by the authors.

Figure 1
Figure 1. Examples of diagnostic assessments designed to evaluate rational decision-making abilities include the Wason [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Larger models, like ChatGPT-4, achieve higher av [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Number of publications in 2024 focusing on en [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

68 extracted references · 57 canonical work pages

  1. [1]

    Constitutional politics/constitutional law

    Bruce Ackerman. Constitutional politics/constitutional law. YAle lJ, 99:453, 1989

    work page 1989

  2. [2]

    Llama 3: Open research and development in large-scale language models

    Meta AI. Llama 3: Open research and development in large-scale language models. Meta AI Research, 2023. Retrieved from https://ai.facebook.com/research/

    work page 2023

  3. [3]

    Mistral: Lightweight and e￿cient open-source language model

    Mistral AI. Mistral: Lightweight and e￿cient open-source language model. Mistral AI Research, 2023. Retrieved from https://www.mistral.ai

    work page 2023

  4. [4]

    lii legal information institute

    David Alexander. lii legal information institute

    work page

  5. [5]

    Evaluating irrationality in large language models and open research questions

    Dana Alsagheer, Rabimba Karanjai, Weidong Shi, Nour Diallo, Yang Lu, Suha Beydoun, and Qiaoning Zhang. Evaluating irrationality in large language models and open research questions. In Proceedings of the HEAL Workshop at CHI . ACM, 2024

    work page 2024

  6. [6]

    Claude 2: An ai assistant with a focus on safety and reliability

    Anthropic. Claude 2: An ai assistant with a focus on safety and reliability. Anthropic AI, 2023. Retrieved from https://www.anthropic.com

    work page 2023

  7. [7]

    The impact of cognitive biases on professionals’ decision- making: A review

    Unknown Author. The impact of cognitive biases on professionals’ decision- making: A review. Unknown Journal, 2023

    work page 2023

  8. [8]

    Barbri Simulated MBE

    Barbri. Barbri Simulated MBE . SIM MBE, 2024

    work page 2024

Show all 68 references
  1. [9]

    Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell

    Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT), pages 610–623, 2021

    2021

  2. [10]

    Pearson correla- tion coe￿cient

    Jacob Benesty, Jingdong Chen, Yiteng Huang, and Israel Cohen. Pearson correla- tion coe￿cient. Noise Reduction in Speech Processing , pages 1–4, 2009

    2009

  3. [11]

    Using cognitive psychology to understand gpt-3

    Marcel Binz and Eric Schulz. Using cognitive psychology to understand gpt-3. Proceedings of the National Academy of Sciences , 120(6):e2218523120, 2023

    2023

  4. [12]

    On the opportunities and risks of foundation models

    Rishi Bommasani et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021

    2021 arXiv

  5. [13]

    Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, and Dario Amodei

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, and Dario Amodei. Language models are few-shot learners. Advances in Neural Information Processing Systems , 33:1877–1901, 2020

    1901

  6. [14]

    Sparks of arti￿cial general intelligence: Early experiments with gpt-4

    Sébastien Bubeck et al. Sparks of arti￿cial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712, 2023

    2023 arXiv

  7. [15]

    Understanding the relationship between rationality and intelligence: a latent-variable approach

    Alexander P Burgoyne, Cody A Mashburn, Jason S Tsukahara, David Z Hambrick, and Randall W Engle. Understanding the relationship between rationality and intelligence: a latent-variable approach. Thinking & Reasoning, 29(1):1–42, 2023

    2023

  8. [16]

    The Measure of Intelligence

    François Chollet. The Measure of Intelligence . arXiv preprint arXiv:1911.01547, 2019

    1911 arXiv

  9. [17]

    Better than rational: Evolutionary psychology and the invisible hand

    Leda Cosmides and John Tooby. Better than rational: Evolutionary psychology and the invisible hand. The American Economic Review, 84:327–332, 1994

    1994

  10. [18]

    Lampinen, Stephanie C

    Ishita Dasgupta, Andrew K. Lampinen, Stephanie C. Y. Chan, Hannah R. Shea- han, Antonia Creswell, Dharshan Kumaran, James L. McClelland, and Felix Hill. Language models show human-like content e￿ects on reasoning tasks, 2024

    2024

  11. [19]

    Gemini 1: Advancing ai capabilities through competitive benchmarks

    DeepMind. Gemini 1: Advancing ai capabilities through competitive benchmarks. DeepMind Research, 2023. Retrieved from https://www.deepmind.com

    2023

  12. [20]

    Towards a rigorous science of interpretable machine learning

    Finale Doshi-Velez and Been Kim. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608, 2017

    2017 arXiv

  13. [21]

    Jonathan St. B. T Evans. Dual-processing accounts of reasoning, judgment, and social cognition. Annual Review of Psychology, 59:255–278, 2008

    2008

  14. [22]

    meaning

    E Allan Farnsworth. " meaning" in the law of contracts. The Yale Law Journal, 76(5):939–965, 1967

    1967

  15. [23]

    Language models do hard arithmetic tasks easily and hardly do easy arithmetic tasks, 2024

    Andrew Gambardella, Yusuke Iwasawa, and Yutaka Matsuo. Language models do hard arithmetic tasks easily and hardly do easy arithmetic tasks, 2024

    2024

  16. [24]

    Fast and frugal heuristics: The tools of bounded rationality

    Gerd Gigerenzer. Fast and frugal heuristics: The tools of bounded rationality. 2004

    2004

  17. [25]

    A survey on large language models: Applications, challenges, limitations, and practical usage

    Muhammad Usman Hadi, Rizwan Qureshi, Abbas Shah, Muhammad Irfan, Anas Zafar, Muhammad Bilal Shaikh, Naveed Akhtar, Jia Wu, Seyedali Mirjalili, et al. A survey on large language models: Applications, challenges, limitations, and practical usage. Authorea Preprints, 2023

    2023

  18. [26]

    Machine psychology: Toward understanding ai cognitive processes through human-centric frameworks

    Thilo Hagendor￿. Machine psychology: Toward understanding ai cognitive processes through human-centric frameworks. AI and Society , 35:423–438, 2020

    2020

  19. [27]

    Thilo Hagendor￿, Ishita Dasgupta, Marcel Binz, Stephanie C. Y. Chan, Andrew Lampinen, Jane X. Wang, Zeynep Akata, and Eric Schulz. Machine psychology, 2024

    2024

  20. [28]

    Criminal Law & Procedure

    Gene Healy. Criminal Law & Procedure . Cato Institute, 2004

    2004

  21. [29]

    The weirdest people in the world? Behavioral and Brain Sciences , 33(2-3):61–83, 2010

    Joseph Henrich, Steven J Heine, and Ara Norenzayan. The weirdest people in the world? Behavioral and Brain Sciences , 33(2-3):61–83, 2010

    2010

  22. [30]

    Real property and real property security: The well-being of the law, 1965

    John R Hetland. Real property and real property security: The well-being of the law, 1965

    1965

  23. [32]

    Training compute- optimal large language models

    Jordan Ho￿mann, Sebastian Borgeaud, Arthur Mensch, et al. Training compute- optimal large language models. arXiv preprint arXiv:2203.15556, 2022

    2022 arXiv

  24. [33]

    Some observations on the law of evidence

    Robert M Hutchins and Donald Slesinger. Some observations on the law of evidence. Colum. L. Rev., 28:432, 1928

    1928

  25. [34]

    Thinking, Fast and Slow

    Daniel Kahneman. Thinking, Fast and Slow . Farrar, Straus and Giroux, 2011

    2011

  26. [35]

    Judgment under uncertainty: Heuristics and biases

    Daniel Kahneman and Amos Tversky. Judgment under uncertainty: Heuristics and biases. Cambridge University Press, 1982

    1982

  27. [36]

    Scaling laws for neural language models

    Jared Kaplan, Sam McCandlish, Tom Henighan, et al. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020

    2001 arXiv

  28. [37]

    Building machines that learn and think like people

    Brenden Lake, Tomer Ullman, and Joshua Tenenbaum. Building machines that learn and think like people. Behavioral and Brain Sciences , 40:e253, 2017

    2017

  29. [38]

    Holistic evaluation of language models

    Percy Liang et al. Holistic evaluation of language models. arXiv preprint arXiv:2211.09110, 2022

    2022 arXiv

  30. [39]

    Cognitive psychology tasks to assess rationality in llms

    Daniel Macmillan-Scott and Mirco Musolesi. Cognitive psychology tasks to assess rationality in llms. Journal of Machine Learning Research , 24(113):1–14, 2023

    2023

  31. [40]

    (ir) rationality and cognitive biases in large language models

    Olivia Macmillan-Scott and Mirco Musolesi. (ir) rationality and cognitive biases in large language models. Royal Society Open Science , 11(6):240255, 2024

    2024

  32. [41]

    The Next Decade in AI: Four Steps Towards Robust Arti￿cial Intelligence

    Gary Marcus and Ernest Davis. The Next Decade in AI: Four Steps Towards Robust Arti￿cial Intelligence. arXiv preprint arXiv:2001.09768, 2020

    2001 arXiv

  33. [42]

    Rebooting AI: Building Arti￿cial Intelligence We Can Trust

    Gary Marcus and Ernest Davis. Rebooting AI: Building Arti￿cial Intelligence We Can Trust. Vintage, 2022

    2022

  34. [43]

    Kevin S. McGrew. The Cattell-Horn-Carroll Theory of Cognitive Abilities: Past, Present, and Future. The Guilford Press, 2009

    2009

  35. [44]

    Rethinking the role of demonstrations and memory in in-context learning

    Sewon Min et al. Rethinking the role of demonstrations and memory in in-context learning. arXiv preprint arXiv:2202.12837, 2022

    2022 arXiv

  36. [45]

    Arti￿cial Intelligence: A Guide to Human-Level Thinking

    Melanie Mitchell. Arti￿cial Intelligence: A Guide to Human-Level Thinking . Pen- guin Random House, 2021

    2021

  37. [46]

    Measuring machine intelligence: A human-centric per- spective

    Melanie Mitchell et al. Measuring machine intelligence: A human-centric per- spective. Communications of the ACM , 66(2):58–65, 2023

    2023

  38. [47]

    Dual process theory of reasoning: Contemporary issues and developmental applications

    Vanessa Moro. Dual process theory of reasoning: Contemporary issues and developmental applications. Developmental Review, 57:100946, 2020

    2020

  39. [48]

    Con￿rmation bias: A ubiquitous phenomenon in many guises

    Raymond S Nickerson. Con￿rmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology , 2(2):175–220, 1998

    1998

  40. [49]

    Culture and systems of thought: holistic versus analytic cognition

    Richard E Nisbett et al. Culture and systems of thought: holistic versus analytic cognition. Psychological Review, 108(2):291, 2001

    2001

  41. [50]

    Bayesian Rationality: The Probabilistic Approach to Human Reasoning

    Mike Oaksford and Nick Chater. Bayesian Rationality: The Probabilistic Approach to Human Reasoning . Oxford University Press, 2007

    2007

  42. [51]

    Gpt-3.5: A compact and e￿cient language model for diverse applications

    OpenAI. Gpt-3.5: A compact and e￿cient language model for diverse applications. OpenAI Research, 2023. Retrieved from https://openai.com

    2023

  43. [52]

    Gpt-4 technical report

    OpenAI. Gpt-4 technical report. https://cdn.openai.com/papers/gpt-4.pdf, 2023. Accessed: 2024-12-09

    2023

  44. [53]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Bal- com, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Je￿...

    2017

  45. [54]

    Training language models to follow instructions with human feedback

    Long Ouyang et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems , 2022

    2022

  46. [55]

    Rational choice and the structure of the environment

    Herbert A Simon. Rational choice and the structure of the environment. Psycho- logical Review, 63(2):129, 1956

    1956

  47. [56]

    Models of bounded rationality: Empirically grounded economic reason, volume 3

    Herbert Alexander Simon. Models of bounded rationality: Empirically grounded economic reason, volume 3. MIT press, 1997

    1997

  48. [57]

    Rationality and the Re￿ective Mind

    Keith E Stanovich. Rationality and the Re￿ective Mind . Oxford University Press, 2011

    2011

  49. [58]

    Stanovich and Richard F

    Keith E. Stanovich and Richard F. West. Individual di￿erences in reasoning: Implications for the rationality debate. Behavioral and Brain Sciences , 23(5):645– 665, 2000

    2000

  50. [59]

    The great rationality debate.Psychological Science, 13(1):94–99, 2002

    Philip E Tetlock and Barbara A Mellers. The great rationality debate.Psychological Science, 13(1):94–99, 2002

    2002

  51. [60]

    Ecological Ratio- nality: Intelligence in the World

    Peter M Todd, Gerd Gigerenzer, and the ABC Research Group. Ecological Ratio- nality: Intelligence in the World . Oxford University Press, 2012

    2012

  52. [61]

    The psychological foundations of culture

    John Tooby and Leda Cosmides. The psychological foundations of culture. The Adapted Mind: Evolutionary Psychology and the Generation of Culture , pages 19–136, 1992

    1992

  53. [63]

    Judgment under uncertainty: Heuristics and biases

    Amos Tversky and Daniel Kahneman. Judgment under uncertainty: Heuristics and biases. Science, 185(4157):1124–1131, 1974

    1974

  54. [64]

    Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment

    Amos Tversky and Daniel Kahneman. Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment. Psychological Review, 90(4):293–315, 1983

    1983

  55. [65]

    Peter C. Wason. Reasoning about a rule. Quarterly Journal of Experimental Psychology, 20:273–281, 1968

    1968

  56. [66]

    Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Je￿ Dean, and William Fe- dus

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Ra￿el, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Je￿ Dean, and William Fe- dus. Emergent abilities of large language models. a...

    2022 arXiv

  57. [67]

    Latest version of chatgpt aces bar exam with score nearing 90th percentile

    Debra Cassens Weiss. Latest version of chatgpt aces bar exam with score nearing 90th percentile. ABA Journal, 16, 2023

    2023

  58. [68]

    Ai in context: Task-speci￿c reasoning and role adaptation

    Jing Wu et al. Ai in context: Task-speci￿c reasoning and role adaptation. Arti￿cial Intelligence Journal, 2022

    2022

  59. [69]

    Explanation by progressive disclosure: An ai-assisted decision- making framework

    Fangchen Xie et al. Explanation by progressive disclosure: An ai-assisted decision- making framework. CHI, 2021

    2021

  60. [70]

    Language models exhibit human-like biases: A study on an- choring and framing e￿ects

    Hao Zhang et al. Language models exhibit human-like biases: A study on an- choring and framing e￿ects. Findings of ACL, 2023. Received 20 February 2007; revised 12 March 2009; accepted 5 June 2009

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.