Pith. sign in

REVIEW 3 major objections 8 minor 1 cited by

Mitigating Spurious Correlations in LLMs via Causality-Aware Post-Training

T0 review · 3 major / 8 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CAPT rewrites the events in a reasoning question as random symbols before fine-tuning, so the model learns the underlying reasoning structure rather than correlations tied to event names.

desk verdict CAPT's random event-symbolization recipe is a genuinely simple and plausible debiasing trick with real OOD gains, but the abstract overclaims ID performance and the event-estimation assumption is never directly tested. read the letter →

arxiv 2506.09433 v1 pith:TLHY7GZL submitted 2025-06-11 cs.LG

Shurui Gui , Shuiwang Ji This is my paper
classification cs.LG
keywords spuriouscorrelationscausalreasoningout-of-distributiongeneralizationeventabstractionsymbolicsupervisedfine-tuningchain-of-thoughtsample-efficient
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that LLM reasoning failures on out-of-distribution inputs are largely spurious correlations at the level of events, not just entities, and that these correlations can be broken after pre-training. It proposes CAPT, which splits the biased prediction into two steps: a pretrained LLM first names the events in the question, then a random symbol assignment erases their identity before the model is fine-tuned. This recoding makes the event variable independent of the reasoning structure, leaving the model to learn only the invariant reasoning trace. The authors report that a 3B-scale model trained with CAPT on 100 in-distribution samples beats both standard fine-tuning and larger models on CLadder and PrOntoQA, including out-of-distribution test sets. The reason the result would matter, if it holds, is that robustness can be bought without collecting hard-to-label out-of-distribution data.

What carries the argument

The machinery is a structural causal model with four roles: $E$ (events or contextual content), $S$ (latent reasoning structure), $X$ (the prompt), and $Y$ (reasoning trace plus answer), with $X$ as a collider and $Y$ dependent on $S$ alone once events are controlled. CAPT is the two-step pipeline built on this graph: a frozen pretrained LLM performs event estimation by mapping the prompt into placeholders $\{\text{symbol}_1,\text{symbol}_2,\dots\}$, and event intervention randomly reassigns those placeholders to capital letters separately for each input. Random assignment is what enforces permutation invariance and blocks new associations during fine-tuning; the frozen estimator never sees the answer, so the spurious path through the confounder is cut.

What would settle it

Annotate an out-of-distribution version of CLadder or PrOntoQA with ground-truth event-to-symbol mappings, run the paper's extraction prompt, and compare the extracted symbols with the ground truth; if extraction accuracy is well below the accuracy the fine-tuned model shows on those same questions, then the assumption that $\mathbb{P}(E|X)$ is unbiased is refuted.

Watch

Extended reading notes

Core claim

The central claim is that the standard learned conditional $\mathbb{P}(Y|X)$ for a reasoning question is contaminated because the prompt $X$ acts as a collider between the event content $E$ and the latent reasoning structure $S$. The paper formalizes this contamination as $$\mathbb{P}(Y|X)=\sum_{e,s}\mathbb{P}(Y|s)\mathbb{P}(s|X)\mathbb{P}(e|X,s)$$ and shows that the last factor is what carries the spurious dependence of the answer $Y$ on events $E$. CAPT removes it in two steps: event estimation extracts $E$ from $X$ using a pretrained LLM, and event intervention rewrites every identified event as a randomly assigned symbol so that $E\perp\!\!\perp S|X$. The result is $$\mathbb{P}(Y|X)=\sum_s \mathbb{P}(Y|s)\mathbb{P}(s|X),$$ a prediction driven only by the reasoning trace, which is what the model is fine-tuned to follow. The same intervention marginalizes out the unobserved confounder $U$ representing pre-training bias.

Load-bearing premise

The claim rests on a pretrained LLM being able to identify the events in a question correctly and without bias, even when the question is out-of-distribution; if that event-estimation step is wrong, the whole symbolization passes the error through and the method's stated sample and model gains do not transfer.

Editorial extensions

If this is right

  • For causal and syllogistic tasks, a 3B model fine-tuned with CAPT on 100 samples is claimed to match or beat larger models that use chain-of-thought prompting, suggesting that scale is not the only route to out-of-distribution robustness.
  • The method needs no out-of-distribution training data: the same randomized symbolization applied at test time maps an OOD question into the same representation space as the in-distribution training samples.
  • Deterministic symbol assignments, such as alphabetic order, converge faster but lead to worse final results, so the randomization step, not symbolization alone, is carrying the fine-tuning debias.
  • When the space of reasoning structures is large, training with chain-of-thought traces is necessary; answer-only supervision can leave the correct reasoning path underdetermined.
  • CAPT's gains are largest precisely on the OOD sets where standard fine-tuning collapses, which is where the paper's separation of pre-training bias from fine-tuning bias makes a concrete practical difference.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If event estimation is transferable, CAPT should extend beyond formal causal and logical benchmarks to any task with identifiable but semantically irrelevant surface features, for example arithmetic word problems or knowledge-grounded question answering; a quick test would be applying the same two-step pipeline to a math word-problem suite with permuted variable names.
  • The method's ceiling is set by the frozen estimator's event-extraction accuracy on shifted inputs; the reported within-3% in-domain drop is not a direct measurement of that error, so measuring extraction error on an OOD set with known ground-truth symbols would predict where CAPT starts to fail.
  • Because only one random assignment is used per input at inference, a natural variant is to sample several assignments and aggregate the model's answers; such ensembling could reduce the variance the authors already observe in converged results and would cheaply test how much of the gain is due to randomization.
  • The argument implies that pre-training bias and fine-tuning bias are separable interventions, which suggests a modular deployment where a stronger frozen event estimator is paired with a small fine-tuned reasoning module and the two are updated independently.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. This paper proposes Causality-Aware Post-Training (CAPT), a fine-tuning strategy for mitigating event-level spurious correlations in LLM reasoning. The authors model data generation as E→X←S→Y, argue that collider bias in this process injects spurious E–Y associations during pre-training via an unobserved confounder U, and propose two steps: event estimation, in which a separate pretrained model (GPT-4o-mini) extracts the events from a prompt under the stated assumption that event estimation is a transferable, bias-diluted capability; and event intervention, in which all events are replaced by randomly assigned alphabet symbols at both training and inference time before SFT of Qwen2.5-3B on 100–200 symbolized samples with chain-of-thought traces. On CLadder and PrOntoQA with commonsense, anti-sense, and non-sense test splits, the paper reports that CAPT reduces the standard deviation across splits and improves OOD accuracy over standard SFT, and in several settings exceeds GPT-4o with CoT prompting without fine-tuning.

Significance. If the transferability assumption holds, CAPT is an attractive and practical recipe: a small model fine-tuned on roughly a hundred samples becomes markedly more robust to event-level shifts, with large, reproducible gains on PrOntoQA anti-sense (e.g., 100-sample CAPT CoT reaches 82.5 versus 70.75 for Original CoT). The paper deserves credit for stating its key assumption explicitly (Section 3.3.1), shipping the full prompts and in-context examples (Appendix C), reporting hyperparameters (Appendix D.1), and including an ablation that isolates the random-assignment component (Figures 3–4), all of which make the method easy to re-implement and to falsify. The comparison against standard SFT is a meaningful external check even though the SCM derivation is best read as a motivating model rather than a proof. The three qualifications that follow — an internal inconsistency in Eqs. (3)–(5), the unmeasured event-estimation accuracy, and an abstract stronger than Table 2 — are all addressable within the manuscript's scope.

major comments (3)
  1. [§3.2, Eqs. (3)–(5); Figure 1(b)–(d)] The derivation mixes direct and indirect effects of the unobserved confounder U without stating the required assumption. Equation (3) factorizes the biased prediction with P(Y|s,u), so U appears as a direct parent of Y, but the red spurious path drawn in Figure 1(b) is X←E←U→S→Y, which contains no U→Y arrow. In Equation (4), the step from Σ_{e,s,u} P(Y,u|s)P(e)P(s)P(X|s,e)/P(X) to Σ_{e,s} P(Y|s)P(s|X)P(e|X,s) is justified only by 'U has been marginalized out'; this is valid only under Y⊥⊥U|S, i.e., the absence of a direct U→Y edge. If such an edge exists, intervening on E (do(E)) removes U→E but leaves U's influence on Y through P(Y|s,u) intact, so the claimed bias elimination does not follow. The paper must either remove u from P(Y|s,u) in Eq. (3) and align the figure, or state the no-direct-effect assumption and explain why it holds for spurious attention patterns.
  2. [§3.3.1, §4.2, Appendix B] The transferability of event estimation is the single point of failure of the entire pipeline, but it is never measured directly. The only validation offered in Section 4.2 (the less-than-3% drop on commonsense sets) is an end-to-end check of estimation + intervention + fine-tuning on in-distribution inputs; it does not give the per-question error rate of GPT-4o-mini's event estimation on anti-sense and non-sense inputs, where event identities are deliberately chosen to conflict with pretraining priors. If an event is omitted, conflated, or mis-mapped on an OOD input, the symbolized prompt contains a corrupted logical structure S, and the OOD gains would then measure the robustness of the symbolization wrapper rather than the causal decomposition. Since CLadder and PrOntoQA are template-generated, gold event sets are available; the paper should report event-estimation accuracy against gold annotations on held-out ID and OOD subsets, plus an error-propagation analysis showing how estimation failures affect final answers. Appendix B's acknowledgment that 'CAPT relies on accurate event estimation' does not substitute for such a measurement.
  3. [Abstract; Table 2] The abstract's claim that with 100 ID fine-tuning samples CAPT enables 3B-scale models to 'outperform both larger LLMs and standard fine-tuning approaches on in-distribution (ID) and OOD tasks' is contradicted by Table 2 in the CLadder Non-sense column: 100-sample CAPT CoT scores 69.43 versus 72.36 for GPT-4o CoT, and the CAPT Answer-only configuration (67.21 ID / 64.42 anti-sense / 63.57 non-sense) is below GPT-4o CoT on all three CLadder columns. The claim should be qualified per dataset and per configuration (e.g., 'CAPT with CoT outperforms standard SFT across all splits and outperforms GPT-4o CoT on PrOntoQA and on the CLadder ID and anti-sense splits'). In addition, the comparison is a system-level one: the CAPT pipeline uses GPT-4o-mini as a non-frozen preprocessing oracle at both training and test time, while the GPT-4o baseline numbers are obtained without symbolization; the text should say this explicitly and, ideally, report the effect of applying the same symbolization to the larger baseline models.
minor comments (8)
  1. [§3.3.2, Eq. (5)] The sentence claiming that random reassignment 'ensures permutation invariance and breaks any selection bias, enforcing the condition E⊥⊥S|X' asserts rather than derives the independence used in Eq. (5); at most, random reassignment can make E and S independent in the empirical training distribution under the additional assumption that the symbolized prompt does not reveal event identity, which is in tension with the fact that event estimation is supposed to recover E from X. Please state the exact independence condition the procedure establishes and prove Eq. (5) from it.
  2. [§3.2, Figure 1(b)] The unobserved confounder U is introduced in Section 3.2 without being part of the SCM defined in Section 3.1; specify the edges incident to U explicitly and reconcile the P(Y|s,u) term in Eq. (3) with the red path X←E←U→S→Y drawn in Figure 1(b).
  3. [Table 2] The label 'Efficient SFT (≤5%)' is ambiguous without dataset totals; state the total training-set sizes for CLadder and PrOntoQA so that '100 ID fine-tuning samples' can be interpreted, and clarify how many samples were used for the full SFT rows.
  4. [§4.2, Table 2] The STD columns report standard deviations across the three test splits, not across fine-tuning runs; since the sample-efficiency comparisons sometimes hinge on small margins, report means and standard deviations over multiple random seeds.
  5. [Appendix D.3, Figures 3–4] The CAPT=order ablation is presented only in figures without converged numeric values or run-to-run variance; report the final accuracies in a table so the magnitude of the random-assignment benefit can be assessed.
  6. [Appendix C.2] In the second CLadder in-context example, the quantity 0.36−0.48 appears once as '−0.13' and once as '−0.12' across the raw and transformed reasoning traces, and it is labeled a 'natural indirect effect' estimand; fix the arithmetic and verify the estimand naming, since these traces serve as the formatting exemplars for event transformation.
  7. [§2] The sentence 'more biased knowledge is also injected into the LLMs during the pertaining' contains a typo ('pertaining' should be 'pre-training').
  8. [References] GPT-4o-mini is cited to the GPT-4 technical report [Achiam et al., 2023], which does not describe that model; cite the appropriate OpenAI model documentation instead.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the core OOD claims are benchmark comparisons against external baselines, and the SCM is an explicit modeling assumption rather than a fitted prediction.

full rationale

The paper's derivation (Sections 3.1-3.3) is built on an explicitly stated SCM, E -> X <- S -> Y, in which event-level information E is declared to be the source of spurious dependence. Equations 2-5 then algebraically show that intervening on E (symbolization plus random reassignment) removes E from the prediction. This is an assumption-based model, not a parameter fitted to the benchmarks, and the key transferability premise (unbiased P(E|X) for pretrained LLMs) is stated as an assumption in Section 3.3.1 rather than derived from the target results. The empirical core is externally benchmarked: CAPT is compared against standard SFT on CLadder and PrOntoQA and against GPT-4o with CoT; those baselines are independent of the CAPT implementation, and the observed failures of plain SFT on anti-sense sets show the comparison could have gone the other way. There are no load-bearing self-citations; the cited SCM/invariance literature and benchmarks are external. One mild self-referential element is that the OOD 'non-sense' sets are constructed by randomizing event names, which is the same operation CAPT applies at inference, so gains on that particular set are partly by construction; however, the anti-sense sets and the separate GPT-4o-mini event-estimation step remain empirical, and the method could fail if event estimation were inaccurate. This is a framing/validation limitation, not a circular derivation, so the score is low.

Assumptions & free parameters 2 free parameters · 5 assumptions · 2 invented entities

The SCM and its independence assumptions are the scaffolding of the method; they are stated, not derived, and the crucial E⊥⊥S|X condition after randomization is assumed to hold.

free parameters (2)
  • Number of in-context examples for event estimation (2 to 3)
    Chosen by hand, not varied or justified; affects symbolization quality and hence the whole pipeline.
  • Random symbol assignment seed
    Not reported; randomization is central to the method, so the seed affects reproducibility.
assumptions (5)
  • domain assumption Data generation follows E→X←S→Y with S as the latent logic structure
    Section 3.1, Figure 1(a); this SCM is the basis of the whole analysis.
  • domain assumption Y is conditionally independent of X and E given S (Y⊥⊥(X,E)|S)
    Used to rewrite P(Y|X) in Eq. 1 and 2; not proven for LLM reasoning traces.
  • ad hoc to paper Pretraining distributions are large and diverse enough to dilute colliding biases in P(E|X)
    Section 3.3.1, the only LLM-specific assumption; load-bearing for event estimation.
  • ad hoc to paper Random assignment of symbols enforces E⊥⊥S|X in the transformed data
    Section 3.3.2, Eq. 5; this conditional independence is what removes the spurious dependence, but it is idealized and not measured.
  • domain assumption CoT traces approximate the latent mediator S
    Section 3.4; used to justify training and inference with CoT.
invented entities (2)
  • Unobserved confounder U
    purpose: Models pre-training spurious correlations between events and answers in the biased model
    Introduced in Figure 1(b) and Eq. 3; no falsifiable handle outside the paper.
  • Latent logic structure S
    purpose: The true reasoning structure that should mediate X and Y
    Postulated as an unobserved variable; the paper's whole analysis rests on it, but it is never directly measured.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mitigating Spurious Correlations in LLMs via Causality-Aware Post-Training." pith.science (2026). https://pith.science/paper/TLHY7GZL

@misc{pith2026250609433,
  author       = {Pith},
  title        = {Pith review of: Mitigating Spurious Correlations in LLMs via Causality-Aware Post-Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TLHY7GZL}},
  note         = {Machine review of arXiv:2506.09433}
}
read the original abstract

While large language models (LLMs) have demonstrated remarkable capabilities in language modeling, recent studies reveal that they often fail on out-of-distribution (OOD) samples due to spurious correlations acquired during pre-training. Here, we aim to mitigate such spurious correlations through causality-aware post-training (CAPT). By decomposing a biased prediction into two unbiased steps, known as \textit{event estimation} and \textit{event intervention}, we reduce LLMs' pre-training biases without incurring additional fine-tuning biases, thus enhancing the model's generalization ability. Experiments on the formal causal inference benchmark CLadder and the logical reasoning dataset PrOntoQA show that 3B-scale language models fine-tuned with CAPT can outperform both traditional SFT and larger LLMs on in-distribution (ID) and OOD tasks using only 100 ID fine-tuning samples, demonstrating the effectiveness and sample efficiency of CAPT.

Figures

Figures reproduced from arXiv: 2506.09433 by the authors.

Figure 1
Figure 1. Acquisitions and eliminations of Spurious correlations. Figure (a) describes the data interaction and generation SCM assumption, where the grey node indicates strong colliding biases such as selection biases. After training models on the data generated, figure (b) indicates the modeling of the confounding correlation when using the pre-trained model. To avoid predicting using the confounding correlation, figures (c)… view at source ↗
Figure 2
Figure 2. CAPT implementation pipeline. The upper part represents the training process, where the event transformation, the orange bold arrow, is illustrated in the lower part of the figure. ability to abstract over event identities and map the large event space into a compact symbolic alphabet. This abstraction facilitates reliable reasoning during both training and inference. 3.3.3 Implementation overview Building on the th… view at source ↗
Figure 3
Figure 3. CAPT ablation study: PrOntoQA. CAPT=null denotes the original SFT performance; CAPT=order indicates deterministic assignments instead of random assignments; CAPT=random is the standard CAPT method [PITH_FULL_IMAGE:figures/full_fig_p022_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: CAPT ablation study: CLadder. CAPT=null denotes the original SFT performance; CAPT=order indicates deterministic assignments instead of random assignments; CAPT=random is the standard CAPT method. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_4.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Reason Before You Retrieve: Agentic Planning for Multi-modal RAG

    cs.AI 2026-06 reject novelty 5.0 of 10

    MM-R2 claims SOTA multimodal RAG accuracy on InfoSeek and Encyclopedic VQA via intent grounding plus a 10-topic KnowledgeMap, but its teacher trajectories leak the gold Wikipedia page and omit the image.

Reference graph

Works this paper leans on

47 extracted references · 13 canonical work pages · cited by 1 Pith paper

  1. [1]

    Language models are greedy reasoners: A systematic formal analysis of chain-of-thought

    Abulhair Saparov and He He. Language models are greedy reasoners: A systematic formal analysis of chain-of-thought. arXiv preprint arXiv:2210.01240, 2022

    arXiv 2022

  2. [2]

    Transformers as soft reasoners over language

    Peter Clark, Oyvind Tafjord, and Kyle Richardson. Transformers as soft reasoners over language. arXiv preprint arXiv:2002.05867, 2020

    arXiv 2002

  3. [3]

    Large language models as commonsense knowledge for large-scale task planning

    Zirui Zhao, Wee Sun Lee, and David Hsu. Large language models as commonsense knowledge for large-scale task planning. Advances in Neural Information Processing Systems, 36: 0 31967--31987, 2023

    work page 2023

  4. [4]

    Large language models can be lazy learners: Analyze shortcuts in in-context learning

    Ruixiang Tang, Dehan Kong, Longtao Huang, and Hui Xue. Large language models can be lazy learners: Analyze shortcuts in in-context learning. arXiv preprint arXiv:2305.17256, 2023

    arXiv 2023

  5. [5]

    Spurious correlations in machine learning: A survey

    Wenqian Ye, Guangtao Zheng, Xu Cao, Yunsheng Ma, and Aidong Zhang. Spurious correlations in machine learning: A survey. arXiv preprint arXiv:2402.12715, 2024

    arXiv 2024

  6. [6]

    Gsm-symbolic: Understanding the limitations of mathematical reasoning in large language models

    Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mehrdad Farajtabar. Gsm-symbolic: Understanding the limitations of mathematical reasoning in large language models. arXiv preprint arXiv:2410.05229, 2024

    arXiv

  7. [7]

    Diab, and Bernhard Sch \"o lkopf

    Zhijing Jin, Jiarui Liu, Zhiheng LYU, Spencer Poff, Mrinmaya Sachan, Rada Mihalcea, Mona T. Diab, and Bernhard Sch \"o lkopf. Can large language models infer causation from correlation? In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=vqIH0ObdqL

    work page 2024

  8. [8]

    A causal view of entity bias in (large) language models

    Fei Wang, Wenjie Mo, Yiwei Wang, Wenxuan Zhou, and Muhao Chen. A causal view of entity bias in (large) language models. arXiv preprint arXiv:2305.14695, 2023

    arXiv 2023

Show all 47 references
  1. [9]

    Entity-based knowledge conflicts in question answering

    Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. Entity-based knowledge conflicts in question answering. arXiv preprint arXiv:2109.05052, 2021

    2021 arXiv

  2. [10]

    Counterfactual inference for text classification debiasing

    Chen Qian, Fuli Feng, Lijie Wen, Chunping Ma, and Pengjun Xie. Counterfactual inference for text classification debiasing. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language...

    2021

  3. [11]

    Cladder: Assessing causal reasoning in language models

    Zhijing Jin, Yuen Chen, Felix Leeb, Luigi Gresele, Ojasv Kamal, LYU Zhiheng, Kevin Blin, Fernando Gonzalez Adauto, Max Kleiman-Weiner, Mrinmaya Sachan, et al. Cladder: Assessing causal reasoning in language models. In Thirty-seventh conference on neural information processing ...

    2023

  4. [12]

    Logic-lm: Empowering large language models with symbolic solvers for faithful logical reasoning

    Liangming Pan, Alon Albalak, Xinyi Wang, and William Yang Wang. Logic-lm: Empowering large language models with symbolic solvers for faithful logical reasoning. arXiv preprint arXiv:2305.12295, 2023

    2023 arXiv

  5. [13]

    Efficient tool use with chain-of-abstraction reasoning

    Silin Gao, Jane Dwivedi-Yu, Ping Yu, Xiaoqing Ellen Tan, Ramakanth Pasunuru, Olga Golovneva, Koustuv Sinha, Asli Celikyilmaz, Antoine Bosselut, and Tianlu Wang. Efficient tool use with chain-of-abstraction reasoning. arXiv preprint arXiv:2401.17464, 2024

    2024 arXiv

  6. [14]

    Modeling the data-generating process is necessary for out-of-distribution generalization

    Jivat Neet Kaur, Emre Kiciman, and Amit Sharma. Modeling the data-generating process is necessary for out-of-distribution generalization. arXiv preprint arXiv:2206.07837, 2022

    2022 arXiv

  7. [15]

    Causal inference by using invariant prediction: identification and confidence intervals

    Jonas Peters, Peter B \"u hlmann, and Nicolai Meinshausen. Causal inference by using invariant prediction: identification and confidence intervals. Journal of the Royal Statistical Society Series B: Statistical Methodology, 78 0 (5): 0 947--1012, 2016

    2016

  8. [16]

    Invariant risk minimization

    Martin Arjovsky, L \'e on Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv:1907.02893, 2019

    1907 arXiv

  9. [17]

    Language models are greedy reasoners: A systematic formal analysis of chain-of-thought

    Abulhair Saparov and He He. Language models are greedy reasoners: A systematic formal analysis of chain-of-thought. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=qFVVBzXxR2V

    2023

  10. [18]

    Improving language understanding by generative pre-training

    Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understanding by generative pre-training. 2018

    2018

  11. [19]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  12. [20]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  13. [21]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  14. [22]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

    arXiv

  15. [23]

    Program induction by rationale generation: Learning to solve and explain algebraic word problems

    Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. Program induction by rationale generation: Learning to solve and explain algebraic word problems. arXiv preprint arXiv:1705.04146, 2017

    2017 arXiv

  16. [24]

    Towards logiglue: A brief survey and a benchmark for analyzing logical reasoning capabilities of language models

    Man Luo, Shrinidhi Kumbhar, Mihir Parmar, Neeraj Varshney, Pratyay Banerjee, Somak Aditya, Chitta Baral, et al. Towards logiglue: A brief survey and a benchmark for analyzing logical reasoning capabilities of language models. arXiv preprint arXiv:2310.00836, 2023

    arXiv

  17. [25]

    Annotation inconsistency and entity bias in multiwoz

    Kun Qian, Ahmad Beirami, Zhouhan Lin, Ankita De, Alborz Geramifard, Zhou Yu, and Chinnadhurai Sankar. Annotation inconsistency and entity bias in multiwoz. arXiv preprint arXiv:2105.14150, 2021 b

    2021 arXiv

  18. [26]

    Scaling laws for neural language models

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020

    2001 arXiv

  19. [27]

    Scaling laws of synthetic data for language models

    Zeyu Qin, Qingxiu Dong, Xingxing Zhang, Li Dong, Xiaolong Huang, Ziyi Yang, Mahmoud Khademi, Dongdong Zhang, Hany Hassan Awadalla, Yi R Fung, et al. Scaling laws of synthetic data for language models. arXiv preprint arXiv:2503.19551, 2025

    2025

  20. [28]

    Evaluating the generalization capabilities of large language models on code reasoning

    Rem Yang, Julian Dai, Nikos Vasilakis, and Martin Rinard. Evaluating the generalization capabilities of large language models on code reasoning. arXiv preprint arXiv:2504.05518, 2025

    2025 arXiv

  21. [29]

    Causal inference in statistics: A primer

    Judea Pearl, Madelyn Glymour, and Nicholas P Jewell. Causal inference in statistics: A primer. John Wiley & Sons, 2016

    2016

  22. [30]

    Bias and fairness in large language models: A survey

    Isabel O Gallegos, Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed. Bias and fairness in large language models: A survey. Computational Linguistics, 50 0 (3): 0 1097--1179, 2024

    2024

  23. [31]

    Some philosophical problems from the standpoint of artificial intelligence

    John McCarthy and Patrick J Hayes. Some philosophical problems from the standpoint of artificial intelligence. In Readings in artificial intelligence, pages 431--450. Elsevier, 1981

    1981

  24. [32]

    Inductive logic programming

    Nada Lavrac and Saso Dzeroski. Inductive logic programming. In WLP, pages 146--160. Springer, 1994

    1994

  25. [33]

    Entailer: Answering questions with faithful and truthful chains of reasoning

    Oyvind Tafjord, Bhavana Dalvi Mishra, and Peter Clark. Entailer: Answering questions with faithful and truthful chains of reasoning. arXiv preprint arXiv:2210.12217, 2022

    2022 arXiv

  26. [34]

    Generating natural language proofs with verifier-guided search

    Kaiyu Yang, Jia Deng, and Danqi Chen. Generating natural language proofs with verifier-guided search. arXiv preprint arXiv:2205.12443, 2022

    2022 arXiv

  27. [35]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35: 0 24824--24837, 2022

    2022

  28. [36]

    Self-consistency improves chain of thought reasoning in language models

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171, 2022

    2022 arXiv

  29. [37]

    Datasets for large language models: A comprehensive survey

    Yang Liu, Jiahuan Cao, Chongyu Liu, Kai Ding, and Lianwen Jin. Datasets for large language models: A comprehensive survey. arXiv preprint arXiv:2402.18041, 2024

    2024 arXiv

  30. [38]

    Do llms have the generalization ability in conducting causal inference? arXiv preprint arXiv:2410.11385, 2024

    Chen Wang, Dongming Zhao, Bo Wang, Ruifang He, and Yuexian Hou. Do llms have the generalization ability in conducting causal inference? arXiv preprint arXiv:2410.11385, 2024

    2024 arXiv

  31. [39]

    Causal parrots: Large language models may talk causality but are not causal

    Matej Ze c evi \'c , Moritz Willig, Devendra Singh Dhami, and Kristian Kersting. Causal parrots: Large language models may talk causality but are not causal. arXiv preprint arXiv:2308.13067, 2023

    2023 arXiv

  32. [40]

    The book of why: the new science of cause and effect

    Judea Pearl and Dana Mackenzie. The book of why: the new science of cause and effect. Basic books, 2018

    2018

  33. [41]

    Zin: When and how to learn invariance without environment partition? Advances in Neural Information Processing Systems, 35: 0 24529--24542, 2022

    Yong Lin, Shengyu Zhu, Lu Tan, and Peng Cui. Zin: When and how to learn invariance without environment partition? Advances in Neural Information Processing Systems, 35: 0 24529--24542, 2022

    2022

  34. [42]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  35. [43]

    Proofwriter: Generating implications, proofs, and abductive statements over natural language

    Oyvind Tafjord, Bhavana Dalvi Mishra, and Peter Clark. Proofwriter: Generating implications, proofs, and abductive statements over natural language. arXiv preprint arXiv:2012.13048, 2020

    2012 arXiv

  36. [44]

    Folio: Natural language reasoning with first-order logic

    Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Wenfei Zhou, James Coady, David Peng, Yujie Qiao, Luke Benson, et al. Folio: Natural language reasoning with first-order logic. arXiv preprint arXiv:2209.00840, 2022

    2022 arXiv

  37. [45]

    On the paradox of learning to reason from data

    Honghua Zhang, Liunian Harold Li, Tao Meng, Kai-Wei Chang, and Guy Van den Broeck. On the paradox of learning to reason from data. arXiv preprint arXiv:2205.11502, 2022

    2022 arXiv

  38. [46]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019

    2019

  39. [47]

    Scikit-learn: Machine learning in python

    Fabian Pedregosa, Ga \"e l Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-learn: Machine learning in python. the Journal of machine Learning research, 12: 0 2825--2...

    2011

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.