Pith. sign in

REVIEW 4 major objections 5 minor 81 references

Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLMs answer familiar causal questions by retrieving memorized knowledge and collapse on fresh 2024-news questions, which this paper takes as evidence that they lack genuine human-like causal reasoning.

desk verdict Useful fresh-benchmark contribution; the "no level-2 causal reasoning" conclusion is not supported by the evidence as presented. read the letter →

arxiv 2506.21215 v1 pith:VFGIT76S submitted 2025-06-26 cs.AI cs.CLcs.LG

Haoang Chi , He Li , Wenjing Yang , Feng Liu , Long Lan , Xiaoguang Ren , Tongliang Liu , Bo Han This is my paper
classification cs.AIcs.CLcs.LG
keywords causalreasoninglargelanguagemodelsProbe-2024training-datacontaminationautoregressivecounterfactualretrieval-augmentedgenerationbenchmarkfreshness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that the causal competence LLMs appear to have is largely a mirage: the models answer familiar causal questions by retrieving causal knowledge memorized from their training data (level-1 causal reasoning) and lack the human-like capacity to derive genuinely new or counterfactual causal knowledge (level-2 causal reasoning). The authors define the two levels explicitly, ground the argument in the autoregressive mechanism of transformer LLMs, and test the claim empirically with a new benchmark, CausalProbe-2024, built from news published after every studied model's training cutoff. All four studied LLMs (LLaMA 2 7B, LLaMA 3 8B, GPT-3.5-turbo, Claude 3 Opus) drop substantially on this fresh benchmark relative to the older COPA, e-CARE, and CausalNet benchmarks, and a membership-inference check supports the claim that the new data are genuinely unseen. If the paper is right, then high scores on established causal QA benchmarks should be read as memory retrieval, and evaluating causal reasoning in LLMs requires fresh, post-cutoff corpora. The paper closes by proposing G²-Reasoner, which injects retrieved general knowledge plus a goal-oriented prompt, and reports partial recovery on fresh and counterfactual questions.

What carries the argument

Three coupled pieces carry the argument. The first is the explicit two-level taxonomy of causal reasoning, with level-1 as fast retrieval of causal knowledge stored in model parameters and context and level-2 as slow derivation of new or unseen causal knowledge, modeled on the fast/slow distinction in cognitive science. The second is the CausalProbe-2024 benchmark itself, whose source articles post-date every studied model's training cutoff, whose Hard tier inserts fabricated cause-effect pairs as counterfactual distractors, and whose freshness is double-checked by the Min-K% Prob membership-inference attack. The third is a causal-graph formalization in which a cause variable $X$ and an effect variable $Y$ are connected through a world-knowledge variable $C$ and emitted as natural-language text $T$; conditioning on the collider $T$ creates the observed association between $X$ and $Y$, which motivates G²-Reasoner's strategy of supplying $P(C)$ externally through retrieval-augmented generation and steering generation with a goal-oriented prompt.

What would settle it

Build a control benchmark from the same post-2024 news corpus that matches COPA's format and per-item difficulty, with simple two-choice cause/effect questions and human-validated answers, and give it to the same four models. If a model scores near its COPA-level accuracy (for instance Claude 3 Opus near 99%) on this matched fresh set, the claim that freshness rather than task difficulty drives the drop would be refuted. A second decisive check is the membership-inference one: if Min-K% Prob on CausalProbe-2024 items turns out comparable to that of older benchmark items, the benchmark is not actually fresher and the empirical pillar collapses.

Watch

Extended reading notes

Core claim

The paper's central claim is that LLMs are confined to level-1 causal reasoning, defined as retrieving causal knowledge embedded in parameters and context, and cannot perform level-2 causal reasoning, defined as deducing new or previously unseen causal knowledge the way humans can. The empirical demonstration is CausalProbe-2024, a three-tier benchmark (Easy, Hard with fabricated false cause-effect distractors, and Multi-choice with a variable number of correct answers) whose corpus consists of news articles published between January and April 2024, after the training cutoffs of all studied LLMs. On this benchmark the studied models show a clear performance drop compared with earlier benchmarks, and the paper attributes the drop to corpus freshness rather than to wording or difficulty, concluding that the autoregressive next-token-prediction objective equips LLMs to reuse familiar causal expressions but not to reason causally about novel situations.

Load-bearing premise

The conclusion stands on the assumption that CausalProbe-2024 exercises the same causal reasoning skill as COPA, e-CARE, and CausalNet, so that the accuracy drop is caused by corpus freshness rather than by harder questions, different formats, or lower-quality answer labels.

Editorial extensions

If this is right

  • Scores on popular causal QA benchmarks such as COPA, e-CARE, and CausalNet should be treated as partly contaminated by training-data overlap, not as clean measures of causal competence.
  • Evaluation of LLM causal reasoning should require corpora published after the model's training cutoff and should verify freshness with membership-inference detection.
  • Autoregressive next-token prediction cannot be cited as evidence of causal ability, because sequential conditioning is compatible with purely memorized causal patterns.
  • G²-Reasoner shows that adding retrieved general knowledge plus a goal-oriented prompt recovers part of the lost accuracy on fresh and counterfactual questions, suggesting level-2-like behavior can be induced without changing the model.
  • On CausalProbe-M, models rarely select false causal statements, so their failure on fresh tests is a miss rate rather than a tendency to invent causal claims.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My reading: the experiments establish a performance gap on one GPT-generated benchmark, but not an absence of capacity; the strongest conclusion the data support is that LLM causal scores are highly sensitive to training-data overlap, with the stronger 'no level-2 capacity' claim going beyond the evidence.
  • A cleaner separation of memorization from reasoning would come from a matched pair of benchmarks, identical in format and validated to be equally difficult, with one topic set from before and one from after the training cutoff, rather than comparison across benchmarks of different ages and formats.
  • The collider-conditioning argument implies an untested corollary: part of the causal signal in text is carried by linguistic structure itself, so models might be improved by exploiting structural cues such as discourse connectives or causal verbs rather than topical knowledge alone.
  • The G²-Reasoner recipe is architecture-agnostic and cheap; its modest gains on a roughly 16 MB knowledge base suggest that scaling or grounding the retrieved knowledge could yield substantially larger level-2-like improvements.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper argues that current LLMs perform only 'level-1' causal reasoning (retrieving familiar causal knowledge from parameters) and lack 'level-2' causal reasoning (deriving new or unseen causal knowledge). The argument has two strands: a methodological claim that autoregressive next-token prediction is not inherently causal, and an empirical claim that LLMs drop sharply on CausalProbe-2024, a new benchmark built from BBC/Guardian articles published after the models' training cut-offs. To improve performance, the paper proposes G2-Reasoner, which combines a small RAG general-knowledge base with a goal-oriented prompt. Experiments across four LLMs and four benchmarks (COPA, e-CARE, CausalNet, CausalProbe-E/H) show a monotonic performance decline as benchmarks become fresher, and G2-Reasoner gives small improvements. The paper also reports a Min-K% Prob freshness check and a human quality-control exercise on a subset of CausalProbe-H.

Significance. The question of whether LLMs genuinely perform causal reasoning or merely reproduce training-data patterns is important and timely, and the paper's proposed benchmark and G2-Reasoner framework are potentially useful resources: the benchmark is open-sourced, the freshness check via Min-K% Prob is a reasonable sanity tool, and the evaluation covers both open- and closed-source models. However, the paper's headline claim—that LLMs 'lack the capacity for genuine human-like (level-2) causal reasoning'—is substantially stronger than what the experiments support. The observed accuracy drops are real as reported, but the cross-benchmark comparison confounds corpus freshness with item format, number of choices, item provenance, and label-generation process. The SCM formalization in Section 5 is not a valid derivation, and the human QC covers only a small subset of one of the three new datasets.

major comments (4)
  1. [Section 4.2 and Table 2] The inference from the accuracy drop to 'the main cause of performance degradation is the freshness of corpora' is confounded. The compared benchmarks differ simultaneously in the number of answer choices (COPA and e-CARE have 2 choices with 50% chance, CausalNet has 3 with 33% chance, CausalProbe-E/H have 4 with 25% chance), in item provenance (human-authored vs. GPT-3.5-generated vs. GPT-4o-mini-generated), and in whether a separately supplied context is provided. Min-K% Prob (Table 3) verifies lower token likelihood for CausalProbe items, but it does not identify corpus freshness as the causal variable; it is a membership-inference heuristic, not a controlled manipulation. A format-matched stale/fresh control (e.g., the same question templates instantiated with pre-2020 and post-2024 articles) or, at minimum, chance-level-adjusted scores and a human exact-match baseline on the full benchmark are needed before the drop can be attributed to freshness.
  2. [Section 5, Eq. (2)] Equation (2) is not a valid marginalization as written. The claimed equality E_{C~P_C} P[Y | X=X0, T=T0, C] = P[Y | X=X0, T=T0] holds only if C is independent of (X, T), which is neither stated nor implied by Figure 4, where C is a confounder. The caveat that 'strictly ensuring the validity of Eq. (2) would require a complete general knowledge base' does not repair the equality; a complete knowledge base would affect the prior P_C and the conditional independence assumptions, but the equation still requires an independence condition that is absent. Moreover, the RAG retrieval used in G2-Reasoner retrieves text similar to the query, which is not equivalent to conditioning on the complete confounder C. The formal justification of G2-Reasoner therefore needs to be either corrected to an explicit conditional-independence argument or honestly reframed as a heuristic motivated by, rather than derived from, causal inference.
  3. [Section 6.1 and Appendix H] The benchmark used to support the central claim was generated by GPT-3.5-turbo (and GPT-4o-mini for CausalProbe-M), i.e., by the same class of system being evaluated. The human quality-control pipeline covers only 260 items from CausalProbe-H (Appendix H), not CausalProbe-E or CausalProbe-M, and the 20-item volunteer screening is too small to calibrate item difficulty. Because the answer labels originate from a GPT model, low LLM accuracy on CausalProbe-2024 could reflect systematic properties of GPT-generated questions or label noise rather than an absence of level-2 causal reasoning. A human exact-match baseline on representative samples of CausalProbe-E and CausalProbe-H, with inter-annotator agreement, is required to support the claim that the benchmark measures causal reasoning rather than generator-specific artifacts.
  4. [Abstract and Definitions 2-3] The categorical conclusion that LLMs 'lack the capacity for genuine human-like (level-2) causal reasoning' exceeds the evidence. The experiments show that the studied LLMs score lower on one GPT-generated fresh benchmark than on older, more familiar benchmarks; they do not demonstrate the absence of level-2 capacity. The autoregressive-training argument in Section 4.1 is an intuitive plausibility argument, not a proof of incapacity, and the paper itself acknowledges that LLMs can sometimes handle unfamiliar texts. The claim should be weakened to a comparative statement (e.g., degraded performance on fresh corpora) or supported by controlled experiments that isolate the capacity in question, such as tasks where parametric knowledge is fully controlled and surface cues cannot be used for memorization.
minor comments (5)
  1. [Section 6.1 and Table captions] Typos and notation: 'Bechmark' should be 'Benchmark', 'represnt' in the Table 2 and Table 3 captions should be 'represent', and 'contruction' in footnote 10 should be 'construction'. Also, Table 2's caption should define 'C-E' and 'C-H' explicitly rather than only in the table body.
  2. [Appendix H] The sentence 'Finally, 232 out of 260 questions were filtered out (temporarily called CausalProbe-HQ), achieving the qualification rate of 89.2%' appears to be a wording error: if 89.2% were qualified, 232 questions should have been retained, not filtered out. Please clarify.
  3. [Figure 9 caption] The caption states that the four benchmarks 'exhibit the similar format and difficulty', but difficulty is precisely the property under test and is not controlled in the comparison. Either remove the 'difficulty' claim or provide evidence that the item difficulties are matched.
  4. [Section 6.3] The observation that 'CoT usually perform a little worse than vanilla' is made without statistical tests or confidence intervals; given the small differences (e.g., 0.001-0.005 in Table 6 for closed-source models) and the use of temperature 1.0 for closed-source models, significance testing or at least a statement about effect sizes is needed.
  5. [Footnote 5] Footnote 5 is confusing: it says COPA/e-CARE/CausalNet contexts are 'not real background knowledge, but a part of the questions themselves', then says in CausalProbe-2024 the 'question' is equivalent to 'context' plus 'question' in the other benchmarks. Clarify the intended distinction between supplying background context and embedding the premise in the question.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular step: the central claim rests on a falsifiable fresh-corpus benchmark and an autoregression argument, neither of which reduces to its own inputs.

full rationale

I walked the paper's claimed derivation chain: the autoregression argument (Section 4.1), the construction and freshness verification of CausalProbe-2024 (Section 6.1, Table 3), the cross-benchmark comparison (Table 2), and the inference that LLMs perform level-1 rather than level-2 causal reasoning (Section 4.2). No load-bearing step reduces to its inputs by construction. The definitions of level-1 and level-2 are used as an interpretive framework, but the empirical test is falsifiable: high performance on fresh corpora would have contradicted the hypothesis, so the conclusion is not self-definitional. The freshness of CausalProbe-2024 is checked with an external membership-inference method (Min-K% Prob) on open-source LLMs, and the paper's limitation section acknowledges that full exclusion of pre-training data cannot be guaranteed. The benchmark was generated with GPT-3.5 turbo, which introduces a potential confound between corpus freshness and item format/difficulty, but that is a correctness and validity concern, not a circular derivation. Appendix H provides independent human checks on a subset, giving external grounding. Because I cannot identify any specific equation, fitted parameter, or self-citation chain that makes the conclusion equivalent to its inputs, the appropriate finding is no significant circularity.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The central claim rests on several unproven premises documented here. No fitted numeric coefficient is used in the main argument, but two hand-picked method choices affect the G2-Reasoner results. The heaviest burden is the interpretive assumption that a performance drop on a fresh, LLM-generated benchmark proves absence of level-2 causal reasoning.

free parameters (2)
  • General knowledge base scale = about 16 MB
    Section 5 and Section 6.2: the authors chose a small Hugging Face general-knowledge dataset; they state a larger Wikipedia API would improve results, so the reported G2-Reasoner gains are tied to this hand-picked capacity choice.
  • Retrieval top-k or similarity threshold
    Appendix C specifies Faiss and Contriever but does not report how many retrieved passages are inserted into the prompt; this unreported choice affects how much retrieved knowledge reaches the model and therefore the G2-Reasoner results.
assumptions (6)
  • ad hoc to paper Autoregressive next-token training implies LLMs can only reuse training-distribution causal patterns and cannot perform level-2 causal reasoning.
    Section 4.1: this is the paper's key methodological premise, supported by an informal argument and a toy example rather than a formal proof.
  • domain assumption The performance drop on CausalProbe-2024 is caused by corpus freshness rather than by benchmark difficulty or construction artifacts.
    Section 4.2 and Section 6.1: no human baseline is provided for CausalProbe-E or CausalProbe-M, so the drop cannot be cleanly separated from inherent task difficulty.
  • domain assumption CausalProbe-2024 is effectively unseen because its source articles postdate the studied model cutoffs; similar events in training data are assumed not to matter.
    Appendix A explicitly admits that the contents are part of human knowledge and that LLMs may have seen comparable information, weakening the unseen claim.
  • domain assumption Min-K% Prob membership inference provides a valid comparison of training-data overlap across non-comparable datasets.
    Table 3 and Appendix E: the metric compares average negative log-likelihood across benchmarks with very different topics and lengths, without calibration or confidence intervals.
  • ad hoc to paper Eq. (2) is valid: conditioning on a complete general knowledge base substitutes for the confounder C in the LLM's causal reasoning posterior.
    Section 5: the authors state that a complete general knowledge base is impractical, so the derivation is invoked as intuitive motivation rather than a proven identity.
  • domain assumption GPT-generated causal QA labels are accurate ground truth for the benchmark.
    Section 6.1 and Appendix H: human validation covered only 260 CausalProbe-H questions, not the full E or M splits, so label noise is not measured for most of the benchmark.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?." pith.science (2026). https://pith.science/paper/VFGIT76S

@misc{pith2026250621215,
  author       = {Pith},
  title        = {Pith review of: Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VFGIT76S}},
  note         = {Machine review of arXiv:2506.21215}
}
read the original abstract

Causal reasoning capability is critical in advancing large language models (LLMs) toward strong artificial intelligence. While versatile LLMs appear to have demonstrated capabilities in understanding contextual causality and providing responses that obey the laws of causality, it remains unclear whether they perform genuine causal reasoning akin to humans. However, current evidence indicates the contrary. Specifically, LLMs are only capable of performing shallow (level-1) causal reasoning, primarily attributed to the causal knowledge embedded in their parameters, but they lack the capacity for genuine human-like (level-2) causal reasoning. To support this hypothesis, methodologically, we delve into the autoregression mechanism of transformer-based LLMs, revealing that it is not inherently causal. Empirically, we introduce a new causal Q&A benchmark called CausalProbe-2024, whose corpora are fresh and nearly unseen for the studied LLMs. The LLMs exhibit a significant performance drop on CausalProbe-2024 compared to earlier benchmarks, indicating the fact that they primarily engage in level-1 causal reasoning. To bridge the gap towards level-2 causal reasoning, we draw inspiration from the fact that human reasoning is usually facilitated by general knowledge and intended goals. We propose G^2-Reasoner, a method that incorporates general knowledge and goal-oriented prompts into LLMs' causal reasoning processes. Experiments demonstrate that G^2-Reasoner significantly enhances LLMs' causal reasoning capability, particularly in fresh and counterfactual contexts. This work sheds light on a new path for LLMs to advance towards genuine causal reasoning, going beyond level-1 and making strides towards level-2.

Figures

Figures reproduced from arXiv: 2506.21215 by the authors.

Figure 1
Figure 1. The motivation of this work. (a) LLMs work well on common causal reasoning tasks, [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. An diagram of illustrating how autoregression fails to capture the correct causal knowledge. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. The causal graph that depicts the data generation mechanism of causal reasoning in LLMs. 4 LLMs cannot Perform Genuine Causal Reasoning In this section, we aim to explore the real causal reasoning capability of current LLMs, in terms of our pre-defined two capability levels: level-1 (Definition 2) and level-2 (Definition 3). We study this problem from both a methodological perspective and an empirical perspective. 4… view at source ↗
Figures from the paper (6 more)
Figure 7
Figure 7. Figure 7: Result comparison be￾tween exact match and partial match 6.2 Implementation Details All the experiments are conducted on the Ubuntu 20.04 system and NVIDIA RTX A6000 GPUs. For closed-source LLMs, i.e., GPT 3.5 turbo and Claude 3 opus, we call the API provided by their …
Figure 8
Figure 8. Figure 8: The pipeline of constructing CausalProbe 2024. CausalProbe 2024 consists of three [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: Examples of four Causal Q&A benchmarks used in this work. These four benchmarks [PITH_FULL_IMAGE:figures/full_fig_p023_9.png]
Figure 11
Figure 11. Figure 11: The statistic of the number of each topic’s data in CausalProbe 2024 [PITH_FULL_IMAGE:figures/full_fig_p023_11.png]
Figure 13
Figure 13. Figure 13: The statis￾tic of query types in each Q&A data of CausalProbe-M. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_13.png]
Figure 14
Figure 14. Figure 14: The prompt templates used in constructing CausalProbe 2024. [PITH_FULL_IMAGE:figures/full_fig_p024_14.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

81 extracted references · 47 canonical work pages

  1. [1]

    Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H

    Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent El Shafey, Yanping Huang, Kathy Meier-Hellstern, Gaurav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Sebastian Ruder, Yi Tay, Kefan Xiao, Yuanzhong Xu, Yujing Z...

    arXiv 2023

  2. [2]

    The claude 3 model family: Opus, sonnet, haiku

    Anthropic. The claude 3 model family: Opus, sonnet, haiku. https: // www. anthropic. com/ news/ claude-3-family, 2024

    2024

  3. [3]

    Self-RAG: Learning to retrieve, generate, and critique through self-reflection

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self-RAG: Learning to retrieve, generate, and critique through self-reflection. InICLR, 2024

    2024

  4. [4]

    Cause and effect: Can large language models truly understand causality?arXiv preprint arXiv:2402.18139, 2024

    Swagata Ashwani, Kshiteesh Hegde, Nishith Reddy Mannuru, Mayank Jindal, Dushyant Singh Sengar, Krishna Chaitanya Rao Kathala, Dishant Banga, Vinija Jain, and Aman Chadha. Cause and effect: Can large language models truly understand causality?arXiv preprint arXiv:2402.18139, 2024

    arXiv 2024

  5. [5]

    GenericsKB: A Knowledge Base of Generic Statements

    Sumithra Bhakthavatsalam, Chloe Anastasiades, and Peter Clark. Genericskb: A knowledge base of generic statements.arXiv preprint arXiv:2005.00660, 2020

    work page Pith review arXiv 2005

  6. [6]

    Causalqa: A benchmark for causal question answering

    Alexander Bondarenko, Magdalena Wolska, Stefan Heindorf, Lukas Blübaum, Axel- Cyrille Ngonga Ngomo, Benno Stein, Pavel Braslavski, Matthias Hagen, and Martin Potthast. Causalqa: A benchmark for causal question answering. InProceedings of the 29th International Conference on Computational Linguistics, pages 3296–3308, 2022

    work page 2022

  7. [7]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwi...

    work page 2020

  8. [8]

    Stephanie C. Y . Chan, Adam Santoro, Andrew K. Lampinen, Jane X. Wang, Aaditya K. Singh, Pierre H. Richemond, James L. McClelland, and Felix Hill. Data distributional properties drive emergent in-context learning in transformers. InNeurIPS, 2022

    work page 2022

Show all 81 references
  1. [9]

    Large language models are visual reasoning coordinators.Advances in Neural Information Processing Systems, 36, 2024

    Liangyu Chen, Bo Li, Sheng Shen, Jingkang Yang, Chunyuan Li, Kurt Keutzer, Trevor Darrell, and Ziwei Liu. Large language models are visual reasoning coordinators.Advances in Neural Information Processing Systems, 36, 2024

    2024

  2. [10]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott G...

    2021 arXiv

  3. [11]

    Lampinen, Stephanie C

    Ishita Dasgupta, Andrew K. Lampinen, Stephanie C. Y . Chan, Hannah R. Sheahan, Antonia Creswell, Dharshan Kumaran, James L. McClelland, and Felix Hill. Language models show human-like content effects on reasoning tasks.arXiv preprint arXiv:2207.07051, 2023

    2023 arXiv

  4. [12]

    A survey on in-context learning.arXiv preprint arXiv:2301.00234, 2023

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, Lei Li, and Zhifang Sui. A survey on in-context learning.arXiv preprint arXiv:2301.00234, 2023

    2023 arXiv

  5. [13]

    The faiss library.arXiv preprint arXiv:2401.08281, 2024

    Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre- Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. The faiss library.arXiv preprint arXiv:2401.08281, 2024

    2024 arXiv

  6. [14]

    e-care: a new dataset for exploring explainable causal reasoning

    Li Du, Xiao Ding, Kai Xiong, Ting Liu, and Bing Qin. e-care: a new dataset for exploring explainable causal reasoning. InACL, 2022

    2022

  7. [15]

    Is ChatGPT a good causal reasoner? a comprehensive evaluation

    Jinglong Gao, Xiao Ding, Bing Qin, and Ting Liu. Is ChatGPT a good causal reasoner? a comprehensive evaluation. InFindings of EMNLP, 2023

    2023

  8. [16]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context.arXiv preprint arXiv:2403.05530, 2024

    Gemini Team Google. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context.arXiv preprint arXiv:2403.05530, 2024

    2024 arXiv

  9. [17]

    Oxford University Press, 04 2007

    Alison Gopnik and Laura Schulz.Causal Learning: Psychology, Philosophy, and Computation. Oxford University Press, 04 2007

    2007

  10. [18]

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y . Wu, Y . K. Li, Fuli Luo, Yingfei Xiong, and Wenfeng Liang. Deepseek-coder: When the large language model meets programming – the rise of code intelligence.arXiv preprint arXiv:240...

    2024 arXiv

  11. [19]

    Cr-lt-kgqa: A knowledge graph question answering dataset requiring commonsense reasoning and long-tail knowledge.arXiv preprint arXiv:2403.01395, 2024

    Willis Guo, Armin Toroghi, and Scott Sanner. Cr-lt-kgqa: A knowledge graph question answering dataset requiring commonsense reasoning and long-tail knowledge.arXiv preprint arXiv:2403.01395, 2024

    2024 arXiv

  12. [20]

    Reasoning with language model is planning with world model

    Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. Reasoning with language model is planning with world model. InEMNLP, 2023

    2023

  13. [21]

    Clarendon Press, 1896

    David Hume.A treatise of human nature. Clarendon Press, 1896

  14. [22]

    Mathprompter: Mathematical reasoning using large language models

    Shima Imani, Liang Du, and Harsh Shrivastava. Mathprompter: Mathematical reasoning using large language models. InACL, 2023

    2023

  15. [23]

    Imbens and Donald B

    Guido W. Imbens and Donald B. Rubin.Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction. Cambridge University Press, 2015

    2015

  16. [24]

    Unsupervised dense information retrieval with contrastive learning

    Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. Unsupervised dense information retrieval with contrastive learning. arXiv preprint arXiv:2112.09118, 2021

    2021 arXiv

  17. [25]

    CLadder: Assessing causal reasoning in language models

    Zhijing Jin, Yuen Chen, Felix Leeb, Luigi Gresele, Ojasv Kamal, Zhiheng Lyu, Kevin Blin, Fer- nando Gonzalez, Max Kleiman-Weiner, Mrinmaya Sachan, and Bernhard Schölkopf. CLadder: Assessing causal reasoning in language models. InNeurIPS, 2023

    2023

  18. [26]

    Diab, and Bernhard Schölkopf

    Zhijing Jin, Jiarui Liu, Zhiheng Lyu, Spencer Poff, Mrinmaya Sachan, Rada Mihalcea, Mona T. Diab, and Bernhard Schölkopf. Can large language models infer causation from correlation? In ICLR, 2024

    2024

  19. [27]

    Kahneman.Thinking, Fast and Slow

    D. Kahneman.Thinking, Fast and Slow. Farrar, Straus and Giroux, 2011. 12

    2011

  20. [28]

    Norm theory: Comparing reality to its alternatives

    Daniel Kahneman and Dale T Miller. Norm theory: Comparing reality to its alternatives. Psychological review, 93(2):136, 1986

    1986

  21. [29]

    A noise audit of human-labeled benchmarks for machine commonsense reasoning.Scientific Reports, 14(1):8609, 2024

    Mayank Kejriwal, Henrique Santos, Ke Shen, Alice M Mulvehill, and Deborah L McGuinness. A noise audit of human-labeled benchmarks for machine commonsense reasoning.Scientific Reports, 14(1):8609, 2024

    2024

  22. [30]

    Causal reasoning and large language models: Opening a new frontier for causality.Transactions on Machine Learning Research, 2024

    Emre Kiciman, Robert Ness, Amit Sharma, and Chenhao Tan. Causal reasoning and large language models: Opening a new frontier for causality.Transactions on Machine Learning Research, 2024

    2024

  23. [31]

    The development of causal reasoning.Wiley Interdisciplinary Reviews: Cognitive Science, 3(3):327–335, 2012

    Deanna Kuhn. The development of causal reasoning.Wiley Interdisciplinary Reviews: Cognitive Science, 3(3):327–335, 2012

    2012

  24. [32]

    Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive NLP tasks. InNeurIPS, 2020

    2020

  25. [33]

    Counterfactual reasoning: Testing language models’ understanding of hypothetical scenarios.arXiv preprint arXiv:2305.16572, 2023

    Jiaxuan Li, Lang Yu, and Allyson Ettinger. Counterfactual reasoning: Testing language models’ understanding of hypothetical scenarios.arXiv preprint arXiv:2305.16572, 2023

    2023 arXiv

  26. [34]

    Gormley, and Jason Eisner

    Chu-Cheng Lin, Aaron Jaech, Xin Li, Matthew R. Gormley, and Jason Eisner. Limitations of autoregressive models and their alternatives. InNAACL-HLT, 2021

    2021

  27. [35]

    Liu, Mohammad Saleh, Etienne Pot, Ben Goodrich, Ryan Sepassi, Lukasz Kaiser, and Noam Shazeer

    Peter J. Liu, Mohammad Saleh, Etienne Pot, Ben Goodrich, Ryan Sepassi, Lukasz Kaiser, and Noam Shazeer. Generating wikipedia by summarizing long sequences. InICLR, 2018

    2018

  28. [36]

    Learning to walk with logical embedding for knowledge reasoning.Information Sciences, 667:120471, 2024

    Ruinan Liu, Guisheng Yin, and Zechao Liu. Learning to walk with logical embedding for knowledge reasoning.Information Sciences, 667:120471, 2024

    2024

  29. [37]

    Large language models and causal inference in collaboration: A comprehensive survey.arXiv preprint arXiv:2403.09606, 2024

    Xiaoyu Liu, Paiheng Xu, Junda Wu, Jiaxin Yuan, Yifan Yang, Yuhang Zhou, Fuxiao Liu, Tianrui Guan, Haoliang Wang, Tong Yu, et al. Large language models and causal inference in collaboration: A comprehensive survey.arXiv preprint arXiv:2403.09606, 2024

    2024 arXiv

  30. [38]

    Causal discovery with language models as imperfect experts.arXiv preprint arXiv:2307.02390, 2023

    Stephanie Long, Alexandre Piché, Valentina Zantedeschi, Tibor Schuster, and Alexandre Drouin. Causal discovery with language models as imperfect experts.arXiv preprint arXiv:2307.02390, 2023

    2023 arXiv

  31. [39]

    Can large language models build causal graphs?arXiv preprint arXiv:2303.05279, 2023

    Stephanie Long, Tibor Schuster, Alexandre Piché, Université de Montreal, ServiceNow Re- search, et al. Can large language models build causal graphs?arXiv preprint arXiv:2303.05279, 2023

    2023 arXiv

  32. [40]

    Insights into llm long-context failures: When transformers know but don’t tell

    Taiming Lu, Muhan Gao, Kuai Yu, Adam Byerly, and Daniel Khashabi. Insights into llm long-context failures: When transformers know but don’t tell. InFindings of EMNLP, 2024

    2024

  33. [41]

    Human language understanding & reasoning.Daedalus, 151(2):127– 138, 2022

    Christopher D Manning. Human language understanding & reasoning.Daedalus, 151(2):127– 138, 2022

    2022

  34. [42]

    Thomas McCoy, Shunyu Yao, Dan Friedman, Matthew Hardy, and Thomas L

    R. Thomas McCoy, Shunyu Yao, Dan Friedman, Matthew Hardy, and Thomas L. Griffiths. Embers of autoregression: Understanding large language models through the problem they are trained to solve.arXiv preprint arXiv:2309.13638, 2023

    2023 arXiv

  35. [43]

    Beyond accuracy: Evaluating the reasoning behavior of large language models – a survey.arXiv preprint arXiv:2404.01869, 2024

    Philipp Mondorf and Barbara Plank. Beyond accuracy: Evaluating the reasoning behavior of large language models – a survey.arXiv preprint arXiv:2404.01869, 2024

    2024 arXiv

  36. [44]

    Prentice- hall Englewood Cliffs, NJ, 1972

    Allen Newell, Herbert Alexander Simon, et al.Human problem solving, volume 104. Prentice- hall Englewood Cliffs, NJ, 1972

    1972

  37. [45]

    GPT-4 technical report.arXiv, 2303.08774, 2023

    OpenAI. GPT-4 technical report.arXiv, 2303.08774, 2023

    2023 arXiv

  38. [46]

    Cambridge University Press, 2 edition, 2009

    Judea Pearl.Causality. Cambridge University Press, 2 edition, 2009

    2009

  39. [47]

    Human and animal cognition: Continuity and discontinuity.Proceedings of the National Academy of Sciences, 104(35):13861–13867, 2007

    David Premack. Human and animal cognition: Continuity and discontinuity.Proceedings of the National Academy of Sciences, 104(35):13861–13867, 2007. 13

    2007

  40. [48]

    Choice of plausible alterna- tives: An evaluation of commonsense causal reasoning

    Melissa Roemmele, Cosmin Adrian Bejan, and Andrew S Gordon. Choice of plausible alterna- tives: An evaluation of commonsense causal reasoning. InAAAI, 2011

    2011

  41. [49]

    CRAB: Assessing the strength of causal relationships between real-world events

    Angelika Romanou, Syrielle Montariol, Debjit Paul, Leo Laugier, Karl Aberer, and Antoine Bosselut. CRAB: Assessing the strength of causal relationships between real-world events. In EMNLP, 2023

    2023

  42. [50]

    Crab: Assessing the strength of causal relationships between real-world events.arXiv preprint arXiv:2311.04284, 2023

    Angelika Romanou, Syrielle Montariol, Debjit Paul, Leo Laugier, Karl Aberer, and Antoine Bosselut. Crab: Assessing the strength of causal relationships between real-world events.arXiv preprint arXiv:2311.04284, 2023

    2023 arXiv

  43. [51]

    Code llama: Open foundation models for code.arXiv preprint arXiv:2308.12950, 2024

    Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, Jérémy Rapin, Artyom Kozhevnikov, Ivan Evtimov, Joanna Bitton, Manish Bhatt, Cristian Canton Ferrer, Aaron Grattafiori, Wenhan Xiong...

    2024 arXiv

  44. [52]

    Leveraging the inductive bias of large language models for abstract textual reasoning

    Christopher Michael Rytting and David Wingate. Leveraging the inductive bias of large language models for abstract textual reasoning. InNeurIPS, 2021

    2021

  45. [53]

    Detecting pretraining data from large language models

    Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. Detecting pretraining data from large language models. InICLR, 2024

    2024

  46. [54]

    Oxford University Press, 08 2005

    Steven Sloman.Causal Models: How People Think about the World and Its Alternatives. Oxford University Press, 08 2005

    2005

  47. [55]

    Towards large language models as copilots for theorem proving in lean.arXiv preprint arXiv:2404.12534, 2024

    Peiyang Song, Kaiyu Yang, and Anima Anandkumar. Towards large language models as copilots for theorem proving in lean.arXiv preprint arXiv:2404.12534, 2024

    2024 arXiv

  48. [56]

    Stanovich and Richard F

    Keith E. Stanovich and Richard F. West. Individual differences in reasoning: Implications for the rationality debate?Behavioral and Brain Sciences, 23(5):645–665, 2000

    2000

  49. [57]

    A causal framework to quantify the robustness of mathematical reasoning with language models

    Alessandro Stolfo, Zhijing Jin, Kumar Shridhar, Bernhard Schölkopf, and Mrinmaya Sachan. A causal framework to quantify the robustness of mathematical reasoning with language models. InACL, 2023

    2023

  50. [58]

    A survey of reasoning with foundation models.arXiv preprint arXiv:2312.11562, 2023

    Jiankai Sun, Chuanyang Zheng, Enze Xie, Zhengying Liu, Ruihang Chu, Jianing Qiu, Jiaqi Xu, Mingyu Ding, Hongyang Li, Mengzhe Geng, et al. A survey of reasoning with foundation models.arXiv preprint arXiv:2312.11562, 2023

    2023 arXiv

  51. [59]

    Can llms learn from previous mistakes? investigating llms’ errors to boost for reasoning.arXiv preprint arXiv:2403.20046, 2024

    Yongqi Tong, Dawei Li, Sizhe Wang, Yujia Wang, Fei Teng, and Jingbo Shang. Can llms learn from previous mistakes? investigating llms’ errors to boost for reasoning.arXiv preprint arXiv:2403.20046, 2024

    2024 arXiv

  52. [60]

    Llama 2: Open foundation and fine-tuned chat models.arXiv, 2307.09288, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, W...

    2023 arXiv

  53. [61]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. InNeurIPS, 2017. 14

    2017

  54. [62]

    A & b== b & a: Triggering logical reasoning failures in large language models.arXiv preprint arXiv:2401.00757, 2024

    Yuxuan Wan, Wenxuan Wang, Yiliu Yang, Youliang Yuan, Jen-tse Huang, Pinjia He, Wenxiang Jiao, and Michael R Lyu. A & b== b & a: Triggering logical reasoning failures in large language models.arXiv preprint arXiv:2401.00757, 2024

    2024 arXiv

  55. [63]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V . Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. InNeurIPS, 2022

    2022

  56. [64]

    Can foundation models talk causality? InUAI 2022 Workshop on Causal Representation Learning, 2022

    Moritz Willig, Matej Zeˇcevi´c, Devendra Singh Dhami, and Kristian Kersting. Can foundation models talk causality? InUAI 2022 Workshop on Causal Representation Learning, 2022

    2022

  57. [65]

    Causality for large language models.arXiv preprint arXiv:2410.15319, 2024

    Anpeng Wu, Kun Kuang, Minqin Zhu, Yingrong Wang, Yujia Zheng, Kairong Han, Baohong Li, Guangyi Chen, Fei Wu, and Kun Zhang. Causality for large language models.arXiv preprint arXiv:2410.15319, 2024

    2024 arXiv

  58. [66]

    Reasoning or reciting? exploring the capabilities and limitations of language models through counterfactual tasks.arXiv preprint arXiv:2307.02477, 2023

    Zhaofeng Wu, Linlu Qiu, Alexis Ross, Ekin Akyürek, Boyuan Chen, Bailin Wang, Najoung Kim, Jacob Andreas, and Yoon Kim. Reasoning or reciting? exploring the capabilities and limitations of language models through counterfactual tasks.arXiv preprint arXiv:2307.02477, 2023

    2023 arXiv

  59. [67]

    An explanation of in-context learning as implicit bayesian inference

    Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. An explanation of in-context learning as implicit bayesian inference. InICLR, 2022

    2022

  60. [68]

    Leandojo: Theorem proving with retrieval-augmented language models.Advances in Neural Information Processing Systems, 36, 2024

    Kaiyu Yang, Aidan Swope, Alex Gu, Rahul Chalamala, Peiyang Song, Shixing Yu, Saad Godil, Ryan J Prenger, and Animashree Anandkumar. Leandojo: Theorem proving with retrieval-augmented language models.Advances in Neural Information Processing Systems, 36, 2024

    2024

  61. [69]

    Towards fine-grained causal reasoning and qa.arXiv preprint arXiv:2204.07408, 2022

    Linyi Yang, Zhen Wang, Yuxiang Wu, Jie Yang, and Yue Zhang. Towards fine-grained causal reasoning and qa.arXiv preprint arXiv:2204.07408, 2022

    2022 arXiv

  62. [70]

    Tree of thoughts: Deliberate problem solving with large language models

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In NeurIPS, 2023

    2023

  63. [71]

    Why do we sometimes get nonsense-correlations between time-series?–a study in sampling and the nature of time-series.Journal of the royal statistical society, 89(1):1–63, 1926

    G Udny Yule. Why do we sometimes get nonsense-correlations between time-series?–a study in sampling and the nature of time-series.Journal of the royal statistical society, 89(1):1–63, 1926

    1926

  64. [72]

    Causal parrots: Large language models may talk causality but are not causal.Transactions on Machine Learning Research, 2023

    Matej Zeˇcevi´c, Moritz Willig, Devendra Singh Dhami, and Kristian Kersting. Causal parrots: Large language models may talk causality but are not causal.Transactions on Machine Learning Research, 2023

    2023

  65. [73]

    Understanding causality with large language models: Feasibility and opportunities.arXiv preprint arXiv:2304.05524, 2023

    Cheng Zhang, Stefan Bauer, Paul Bennett, Jiangfeng Gao, Wenbo Gong, Agrin Hilmkil, Joel Jennings, Chao Ma, Tom Minka, Nick Pawlowski, and James Vaughan. Understanding causality with large language models: Feasibility and opportunities.arXiv preprint arXiv:2304.05524, 2023

    2023 arXiv

  66. [74]

    Situatedgen: Incorporating geographical and temporal contexts into generative commonsense reasoning.Advances in Neural Information Processing Systems, 36, 2024

    Yunxiang Zhang and Xiaojun Wan. Situatedgen: Incorporating geographical and temporal contexts into generative commonsense reasoning.Advances in Neural Information Processing Systems, 36, 2024

    2024

  67. [75]

    Archer: A human-labeled text-to-sql dataset with arithmetic, commonsense and hypothetical reasoning.arXiv preprint arXiv:2402.12554, 2024

    Danna Zheng, Mirella Lapata, and Jeff Z Pan. Archer: A human-labeled text-to-sql dataset with arithmetic, commonsense and hypothetical reasoning.arXiv preprint arXiv:2402.12554, 2024

    2024 arXiv

  68. [76]

    Can large language models distinguish cause from effect? InUAI 2022 Workshop on Causal Representation Learning, 2022

    LYU Zhiheng, Zhijing Jin, Rada Mihalcea, Mrinmaya Sachan, and Bernhard Schölkopf. Can large language models distinguish cause from effect? InUAI 2022 Workshop on Causal Representation Learning, 2022

    2022

  69. [77]

    Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?

    Yu Zhou, Xingyu Wu, Beicheng Huang, Jibin Wu, Liang Feng, and Kay Chen Tan. Causalbench: A comprehensive benchmark for causal learning capability of large language models.arXiv preprint arXiv:2404.06349, 2024. 15 Supplement to “Unveiling Causal Reasoning in Large Language Mode...

    2024 arXiv

  70. [78]

    [59] shows that LLMs can learn from mistakes in logical reasoning

    propose an automatic evaluation framework for LLMs’ logical reasoning ability, and find LLMs have difficulty in performing logical reasoning well. [59] shows that LLMs can learn from mistakes in logical reasoning. Causal reasoning is the process of identifying and understandin...

  71. [79]

    If cats were vegetarians, what results would happen?

    indicates that LLMs’ expertise may contain errors, which could undermine causal reasoning. Therefore, it’s essential to incorporate external expert knowledge into LLMs to enhance LLMs’ causal reasoning abilities. Counterfactual reasoning is an essential task of causal reasonin...

  72. [80]

    [69] collect data from Yahoo and employ crowd workers to generate the causal questions

    is a human-labeled dataset that includes 24K question-answering pairs. [69] collect data from Yahoo and employ crowd workers to generate the causal questions. However, the data they collected are between December 2020 and July 2021, which may be included in the pre-training co...

    2020

  73. [2024]

    Vanilla” denotes doing inference directly. “C-E

    Specifically, we randomly sampled 260 questions from CausalProbe-H and assigned each question to 3 volunteers randomly, using the Algorithm 1. Each volunteer received a total of 60 questions. After receiving their feedback, we treated those questions correctly answered by no l...

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.