Pith. sign in

REVIEW 2 major objections 5 minor 44 references

Propositional Logic for Probing Generalization in Neural Networks

T0 review · 2 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that standard Transformers cannot learn a single, operator-independent negation rule from data alone: when training never pairs NOT with a particular operator, they fail to apply negation to that operator at test time.

desk verdict Stark, well-designed result showing Transformers fail to compose negation with unseen operators, but a missing DeMorgan-equivalent control should be addressed before the claim is fully sealed. read the letter →

arxiv 2506.08978 v1 pith:5HWORWWR submitted 2025-06-10 cs.LG cs.AI

Anna Langedijk , Jaap Jumelet , Willem Zuidema This is my paper
classification cs.LGcs.AI
keywords compositionalgeneralizationnegationpropositionallogicTransformersgraphconvolutionalnetworksLSTMsatisfyingassignmentsinductivebias
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether neural sequence models trained on propositional logic actually learn the rules, by withholding one operator combination from training and testing whether the model can still use it. The authors build a balanced version of an existing satisfiable-formula dataset, define seven unseen-pattern splits (for example, NOT applied to OR, AND, or XOR), and train Transformers, a graph convolutional network, and an LSTM to output satisfying assignments. All models perform well in-distribution, but when the withheld pattern is a negated binary operator, generalization collapses: vanilla Transformers score near chance on formulas containing that pattern, and no architecture reliably solves $\neg(a\lor b)$ when NOT-OR was never seen. If the result is right, training data alone does not give Transformers a single operator-independent negation rule, which matters for any claim that sequence models learn compositional reasoning from examples.

What carries the argument

The load-bearing instrument is the unseen-pattern split built on a rebalanced dataset. Starting from the original randomly generated satisfying assignments, the authors rotate subtrees so that left- and right-branching trees occur equally, eliminating a branching-direction shortcut present in the source data. They then either rewrite or delete every formula containing one chosen parent-child pattern, so that a logical structure is absent from training while the semantic expressivity of the formulas is preserved. A templated test set of 8,301 short formulas places each held-out pattern in controlled contexts, and a behavioural probe feeds the model $\phi$ in place of $\neg\phi$ and classifies the output as correct, unchanged (negation ignored), a different valid answer, or wrong. Together these components separate an inability to combine specific operators from an inability to solve the formula.

What would settle it

Run a vanilla Transformer on the split where NOT-OR is never seen and test it on the template sentence $\neg(a\lor b)$: if it outputs a satisfying assignment such as $a=0,b=0$ at a rate clearly above chance, the claim that it cannot compose negation with an unseen operator is falsified. A complementary check is to compare the distribution of possible satisfying outputs on held-out-pattern formulas with the same distribution on the rewritten formulas used to remove those patterns; a mismatch there would show the split changed task difficulty rather than isolating compositional generalization.

Watch

Extended reading notes

Core claim

The paper's central claim is that the gap is specific to negation rather than to compositional generalization in general. On a rebalanced dataset, Transformers reach about 94% semantic accuracy and GCN and LSTM encoders about 87%, but those aggregate scores hide a sharp split in how the models generalize to seven withheld parent-child operator patterns. When the withheld pattern involves negation applied to one of the binary operators (NOT-OR, NOT-AND, NOT-XOR), the models fail at or near chance on formulas that contain it, and a behavioural test that replaces $\neg\phi$ with $\phi$ shows that vanilla Transformers often leave their output unchanged, that is, the negation is ignored rather than composed. Patterns that withhold other combinations, such as AND with an XOR child or IFF with a negated child, are generalized successfully. The authors conclude that Transformers learn generalizable representations of variables and of the binary operators, but fail to form a unified representation of negation; tree-based encodings, graph structure, and recurrence improve two of the three negation patterns but do not restore full systematicity.

Load-bearing premise

The diagnosis depends on the assumption that removing a pattern from training does not secretly change how easy the test outputs are to guess, so that the only real difference between seen and held-out formulas is the input's structure; if a rewrite or removal alters the output distribution, or if a logically equivalent formula leaks the missing pattern through a shortcut, low accuracy on the held-out patterns would not specifically indict a missing negation rule.

Editorial extensions

If this is right

  • High in-distribution accuracy on logic formulas is not evidence that a model has learned the underlying rules: a model can score near-perfect on the base task and still fail at chance level when a negation-operator pairing is reintroduced.
  • For Transformers, seeing a token such as NOT followed by every operator during training is apparently necessary for composing negation; benchmarks that claim systematic reasoning should therefore control for which operator pairs appear in training data.
  • Structural inductive biases, namely tree positional encodings, graph convolution, and recurrence, partially compensate for missing negation patterns, so architecture choice, not just data volume, is what determines whether a compositional operation is learned.
  • The failure is not a general inability to handle unseen combinations: patterns that withhold non-negation parent-child pairs (AND-XOR, IFF-negation, AND-negation) generalize successfully, localizing the deficit to negation as an operator.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One extension the paper leaves implicit is to test whether the negation deficit is driven by token-level co-occurrence statistics: adding filler examples that contain the NOT-operator bigram in irrelevant, easily solvable contexts should improve the withheld-pattern scores if co-occurrence is the binding constraint, and should not if the model needs the full operator semantics.
  • A testable consequence for large language models is that negation errors should concentrate where the negation token rarely co-occurs with the surrounding construction in training text; this can be checked with corpus frequencies alone, without retraining.
  • The 'ignore the NOT' behavioural probe could be turned into a continuous diagnostic for other models: measure how often replacing $\neg\phi$ with $\phi$ leaves the prediction unchanged, and use that rate as an operational measure of whether an architecture treats negation as a general, composable operator.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper investigates generalization of Transformer, GCN, and LSTM encoder-decoder models on a sequence-to-sequence task that maps propositional formulas in Polish notation to satisfying variable assignments. The authors introduce a balanced version of the Hahn et al. dataset and construct seven training splits, each omitting one direct parent-child operator pattern. Their main finding is that all architectures perform well in-distribution, and while most held-out patterns are generalized successfully, patterns in which negation is applied to a binary operator (P1: not-and, P2: not-or, P3: not-xor) cause large accuracy drops when omitted from training. The paper interprets this as evidence that vanilla Transformers, in particular, fail to learn a unified, operator-independent representation of negation.

Significance. If the empirical claims hold, this is a valuable controlled contribution to the compositionality literature: it shows that the failure is specific to negated binary operators rather than a general inability to combine operators, and it provides a clean benchmark (Prop35Balanced plus the seven splits) for future work. The semantic evaluation against an external SAT solver is a strength, as is the behavioral analysis that distinguishes ignoring negation from other error types. The main limitation is the missing control on the equivalent rewritten forms, which leaves open an alternative explanation for the P1-P3 failures. The paper does not currently ship code or data, which further limits verification.

major comments (2)
  1. [§5.1 and §6] The held-out models for P1-P3 are never evaluated on the DeMorgan-equivalent rewrite forms that define the training split. Specifically, the P1-omitted model should be tested on formulas of the form | ! A ! B, the P2-omitted model on & ! A ! B, and the P3-omitted model on <-> A B. If a model trained without P1 also fails on | ! A ! B, the low accuracy on the original ! & A B pattern cannot be attributed to a missing compositional negation operator, because the rewrite procedure itself may have disrupted learning of the relevant semantics. If it succeeds on the equivalents, the negative result is cleanly attributable to the unseen surface pattern. This control is necessary to support the paper's central conclusion that Transformers fail to apply negation compositionally.
  2. [§5] The blanket statement that omitting any of the seven patterns 'does not reduce the semantic expressivity seen during training' is supported only for P1-P3, which are explicitly rewritten with DeMorgan-equivalent forms. For P4, P6, and P7 the paper states that sentences are 'simply remove[d]' from the training data, and no constructive equivalence is given to show that the same set of truth functions remains expressible. Because the experiment is designed to hold output difficulty constant, the authors should either provide explicit rewrite rules or a proof of expressivity preservation for the removal-based splits, or qualify the claim so that it applies only to the patterns for which it is verified.
minor comments (5)
  1. [Abstract and §7] The abstract's claim that Transformers fail to apply negation compositionally 'unless structural biases are introduced' is contradicted by the P1 results, where GCN and LSTM encoders also fail (Figure 5); the conclusion should be revised to state that structural biases help for some patterns (P2, P3) but not for P1.
  2. [§5.1] There is a typo in the sentence 'pattern (P1) is be rewritten to | ! A ! B'; it should read 'is rewritten'.
  3. [Table 5] The 'Correct?' column uses 'Y' and 'N' without a definition; a legend or table caption should explain these symbols.
  4. [General] The paper does not provide a link to the dataset or training code, even though the central results are empirical and depend on the exact split construction; a public release would be needed for reproducibility.
  5. [§6.1] The behavioral categories (B) and (C) are not mutually exclusive as defined; 'Does not change its prediction' and 'Changes its prediction to an alternative output for the original sentence' can overlap, so the classification should be defined more precisely.

Circularity Check

0 steps flagged · score 1.0 of 10

No circularity: generalization claim rests on external SAT-solver ground truth and independent base-model controls; self-citations are background only.

full rationale

The paper is an empirical study whose central claim (vanilla Transformers fail to compose negation with unseen binary operators) is established by training on splits that rewrite P1–P3 into logically equivalent DeMorgan forms and then testing on the original held-out surface patterns. This is not a derivation: no parameter is fitted to the target result, no defining equation equates the conclusion with an input, and no uniqueness theorem from the authors' prior work is invoked. Ground-truth assignments come from the external Glucose/pyaiger SAT solver, and the base models (trained on all patterns) achieve ~98–100% on the same P1–P3 test subsets, so the failure cannot be attributed to the inherent difficulty of the test sentences. The acknowledged limitations (e.g., no UNSAT formulas, limited hyperparameter search for non-Transformer encoders) are transparent experimental caveats, not circularity. The skeptic's point that held-out models are not tested on the rewritten equivalents is a potential control concern about whether the failure reflects surface-form distribution shift; however, the paper reports that performance on sentences not containing the held-out pattern is unaffected for all models, so the models do solve the in-distribution rewritten forms. The only self-citations (refs 1, 15, 16) are background on inductive biases and interpretability and are not load-bearing for the generalization claim. No circular step can be exhibited by quote and reduction, so the appropriate finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

This is an empirical study; the central claim rests on the validity of the dataset construction and the evaluation protocol rather than on a mathematical derivation. We list the domain assumptions that, if false, would confound the interpretation. No fitted constants or invented entities are used.

assumptions (4)
  • domain assumption The task of generating a satisfying partial assignment is a valid measure of whether a model understands propositional logic.
    The paper equates semantic accuracy on this task with logical understanding (Sections 1 and 4.4). If the task were solvable by surface heuristics, the generalization conclusions would be weakened.
  • domain assumption The Prop35Balanced dataset is generated by flipping 50% of subtrees in the original Hahn et al. formulas, preserving satisfiability and keeping the ground-truth target sequence, so the rebalanced dataset is semantically valid.
    Described in Appendix A.1. The claim that training on balanced data does not hurt accuracy (Table 6) supports this, but the validity of the rewriting itself is assumed.
  • standard math De Morgan's laws and double-negation elimination are valid rewrite rules for creating semantically equivalent formulas.
    Used in Section 5.1 to construct splits P1-P3 via rewriting; this is standard propositional logic.
  • domain assumption Removing or rewriting formulas containing the held-out pattern does not change the distribution of possible target outputs, so performance differences are due to the input pattern alone.
    Stated in Section 5 ('it is only the input that changes'). If false, the controlled comparison would be confounded by output-distribution shift.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Propositional Logic for Probing Generalization in Neural Networks." pith.science (2026). https://pith.science/paper/5HWORWWR

@misc{pith2026250608978,
  author       = {Pith},
  title        = {Pith review of: Propositional Logic for Probing Generalization in Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5HWORWWR}},
  note         = {Machine review of arXiv:2506.08978}
}
read the original abstract

The extent to which neural networks are able to acquire and represent symbolic rules remains a key topic of research and debate. Much current work focuses on the impressive capabilities of large language models, as well as their often ill-understood failures on a wide range of reasoning tasks. In this paper, in contrast, we investigate the generalization behavior of three key neural architectures (Transformers, Graph Convolution Networks and LSTMs) in a controlled task rooted in propositional logic. The task requires models to generate satisfying assignments for logical formulas, making it a structured and interpretable setting for studying compositionality. We introduce a balanced extension of an existing dataset to eliminate superficial patterns and enable testing on unseen operator combinations. Using this dataset, we evaluate the ability of the three architectures to generalize beyond the training distribution. While all models perform well in-distribution, we find that generalization to unseen patterns, particularly those involving negation, remains a significant challenge. Transformers fail to apply negation compositionally, unless structural biases are introduced. Our findings highlight persistent limitations in the ability of standard architectures to learn systematic representations of logical operators, suggesting the need for stronger inductive biases to support robust rule-based reasoning.

Figures

Figures reproduced from arXiv: 2506.08978 by the authors.

Figure 1
Figure 1. Illustration of the model architecture and task. The decoder is a standard Transformer model, for the encoder we [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Generalization on the templated test set, for all architectures. Each model (except the base models) is evaluated on [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. Unchanged predictions, per model, per pattern. Results are shown on the templated test set (top row) and PropRan [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Distribution of possible worlds in the validation set. [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]
Figure 5
Figure 5. Figure 5: Generalization for all model architectures, on the templated test set. Results are averaged over model seeds. Perfor [PITH_FULL_IMAGE:figures/full_fig_p021_5.png]
Figure 6
Figure 6. Figure 6: Generalization on the normal test set, for all architectures. Each model (except the base models) is evaluated on [PITH_FULL_IMAGE:figures/full_fig_p022_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

44 extracted references · 20 canonical work pages

  1. [1]

    Samira Abnar, Mostafa Dehghani, and Willem H. Zuidema. 2020. Transferring Inductive Biases through Knowledge Distillation.CoRRabs/2006.00555 (2020). arXiv:2006.00555 https://arxiv.org/abs/2006.00555

    arXiv 2020

  2. [2]

    Gilles Audemard and Laurent Simon. 2018. On the Glucose SAT Solver.International Journal on Artificial Intelligence Tools27, 01 (Feb. 2018), 1840001. https://doi.org/10.1142/S0218213018400018

    doi:10.1142/s0218213018400018 2018

  3. [3]

    Arthur Conmy, Augustine Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga-Alonso. 2023. Towards automated circuit discovery for mechanistic interpretability.Advances in Neural Information Processing Systems36 (2023), 16318–16352

    work page 2023

  4. [4]

    Lamb, Leo de Penning, B

    Artur d’Avila Garcez, Sebastian Bader, Howard Bowman, Luis C. Lamb, Leo de Penning, B. V . Illuminoo, and Hoifung Poon

    work page

  5. [5]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai D...

    arXiv doi:10.48550/arxiv.2501.12948 2025

  6. [6]

    Richard Evans, David Saxton, David Amos, Pushmeet Kohli, and Edward Grefenstette. 2018. Can Neural Networks Understand Logical Entailment?6th International Conference on Learning Representations, ICLR 2018 - Conference Track Proceedings (2018), 1–15

    work page 2018

  7. [7]

    Fodor and Zenon W

    Jerry A. Fodor and Zenon W. Pylyshyn. 1988. Connectionism and cognitive architecture: A critical analysis.Cognition28, 1 (1988), 3–71. https://doi.org/10.1016/0010-0277(88)90031-5

    doi:10.1016/0010-0277(88)90031-5 1988

  8. [8]

    Kreber, Markus N

    Christopher Hahn, Frederik Schmitt, Jens U. Kreber, Markus N. Rabe, and Bernd Finkbeiner. 2021. Teaching Temporal Logics to Neural Networks. InInternational Conference on Learning Representations. Virtual Event, Austria, May 3-7, 2021. https://openreview.net/forum?id=dOcQK-f4byz Preprint, Vol. 1, Article . Publication date: September 2025. 16•Anna Langedi...

    work page 2021

Show all 44 references
  1. [9]

    Yiding Hao, Dana Angluin, and Robert Frank. 2022. Formal Language Recognition by Hard Attention Transformers: Perspectives from Circuit Complexity.Transactions of the Association for Computational Linguistics10 (July 2022), 800–810. https://doi.org/10.1162/tacl_a_00490

    2022 doi

  2. [10]

    Rishi Hazra, Gabriele Venturato, Pedro Zuidberg Dos Martires, and Luc De Raedt. 2025. Have Large Language Models Learned to Reason? A Characterization via 3-SAT Phase Transition. https://doi.org/10.48550/arXiv.2504.03930 arXiv:2504.03930 [cs]

    2025 doi arXiv

  3. [11]

    Chadi Helwe, Chloe Clavel, and Fabian Suchanek. 2021. Reasoning with Transformer-based Models: Deep Learning, but Shallow Reasoning. In3rd Conference on Automated Knowledge Base Construction. https://doi.org/10.24432/C5W300

    2021 doi

  4. [12]

    Md Mosharaf Hossain, Venelin Kovatchev, Pranoy Dutta, Tiffany Kao, Elizabeth Wei, and Eduardo Blanco. 2020. An Analysis of Natural Language Inference Benchmarks through the Lens of Negation. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processi...

    2020 doi

  5. [13]

    Dieuwke Hupkes, Verna Dankers, Mathijs Mul, and Elia Bruni. 2020. Compositionality Decomposed: How Do Neural Networks Generalise?Journal of Artificial Intelligence Research67 (2020). https://doi.org/10.1613/jair.1.11674

    2020 doi

  6. [14]

    Dieuwke Hupkes, Mario Giulianelli, Verna Dankers, Mikel Artetxe, Yanai Elazar, Tiago Pimentel, Christos Christodoulopoulos, Karim Lasri, Naomi Saphra, Arabella Sinclair, Dennis Ulmer, Florian Schottmann, Khuyagbaatar Batsuren, Kaiser Sun, Kous- tuv Sinha, Leila Khalatbari, Mar...

    2022 arXiv

  7. [15]

    Dieuwke Hupkes and Willem Zuidema. 2018. Visualisation and ’diagnostic Classifiers’ Reveal How Recurrent and Recursive Neural Networks Process Hierarchical Structure.IJCAI International Joint Conference on Artificial Intelligence2018-July (2018), 5617–5621

    2018

  8. [16]

    Jaap Jumelet and Willem Zuidema. 2023. Transparency at the Source: Evaluating and Interpreting Language Models With Access to the True Distribution. arXiv:2310.14840 [cs] http://arxiv.org/abs/2310.14840

    2023 arXiv

  9. [17]

    Nora Kassner and Hinrich Schütze. 2020. Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly. arXiv:1911.03343 [cs] http://arxiv.org/abs/1911.03343

    2020 arXiv

  10. [18]

    Najoung Kim and Tal Linzen. 2020. COGS: A Compositional Generalization Challenge Based on Semantic Interpretation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Online, 9087–9105. h...

    2020 doi

  11. [19]

    Kipf and Max Welling

    Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. https: //doi.org/10.48550/arXiv.1609.02907 arXiv:1609.02907 [cs, stat]

    2017 doi arXiv

  12. [20]

    Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. 2019. Similarity of neural network representations revisited. InInternational conference on machine learning. PMLR, 3519–3529

    2019

  13. [21]

    Brenden Lake and Marco Baroni. 2018. Generalization without Systematicity: On the Compositional Skills of Sequence-to- Sequence Recurrent Networks. In35th International Conference on Machine Learning, ICML 2018, Vol. 7. 4487–4499

    2018

  14. [22]

    Lake, Tomer D

    Brenden M. Lake, Tomer D. Ullman, Joshua B. Tenenbaum, and Samuel J. Gershman. 2016. Building Machines That Learn and Think Like People.Behavioral and Brain Sciences2012 (2016), 1–101. https://doi.org/10.1017/S0140525X16001837

    2016 doi

  15. [23]

    Junteng Liu, Yuanxiang Fan, Zhuo Jiang, Han Ding, Yongyi Hu, Chi Zhang, Yiqi Shi, Shitong Weng, Aili Chen, Shiqi Chen, Yunan Huang, Mozhi Zhang, Pengyu Zhao, Junjie Yan, and Junxian He. 2025. SynLogic: Synthesizing Verifiable Reasoning Data at Scale for Learning Logical Reason...

    2025 arXiv

  16. [24]

    João Loula, Marco Baroni, and Brenden Lake. 2019. Rearranging the Familiar: Testing Compositional Generalization in Recurrent Networks. (2019), 108–114. https://doi.org/10.18653/v1/w18-5413

    2019 doi

  17. [25]

    G.F. Marcus. 2003.The Algebraic Mind: Integrating Connectionism and Cognitive Science. MIT Press. https://books.google.nl/ books?id=7YpuRUlFLm8C

    2003

  18. [26]

    Gary F. Marcus. 1998. Rethinking Eliminative Connectionism.Cognitive Psychology37, 3 (1998), 243–282. https://doi.org/10. 1006/cogp.1998.0694

    1998

  19. [27]

    Thomas McCoy, Ellie Pavlick, and Tal Linzen

    R. Thomas McCoy, Ellie Pavlick, and Tal Linzen. 2019. Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference.arXiv2 (2019), 3428–3448

    2019

  20. [28]

    Philipp Mondorf and Barbara Plank. 2024. Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models – A Survey. arXiv:2404.01869 [cs] http://arxiv.org/abs/2404.01869

    2024 arXiv

  21. [29]

    Mihir Parmar, Nisarg Patel, Neeraj Varshney, Mutsumi Nakamura, Man Luo, Santosh Mashetty, Arindam Mitra, and Chitta Baral. 2024. LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models. In Proceedings of the 62nd Annual Meeting of the As...

    2024 doi

  22. [30]

    Kyle Richardson and Ashish Sabharwal. 2021. Pushing the Limits of Rule Reasoning in Transformers through Natural Language Satisfiability. arXiv:2112.09054 [cs] http://arxiv.org/abs/2112.09054 Preprint, Vol. 1, Article . Publication date: September 2025. Propositional Logic for...

    2021 arXiv

  23. [31]

    Daniel Selsam, Matthew Lamm, Benedikt Bünz, Percy Liang, Leonardo de Moura, and David L. Dill. 2019. Learning a SAT Solver from Single-Bit Supervision. arXiv:1802.03685 [cs] http://arxiv.org/abs/1802.03685

    2019 arXiv

  24. [32]

    She, Christopher Potts, Samuel R

    Jingyuan S. She, Christopher Potts, Samuel R. Bowman, and Atticus Geiger. 2023. ScoNe: Benchmarking Negation Reasoning in Language Models With Fine-Tuning and In-Context Learning. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume...

    2023 doi

  25. [33]

    Vighnesh Leonardo Shiv and Chris Quirk. 2019. Novel Positional Encodings to Enable Tree-Based Transformers. (2019), 11

    2019

  26. [34]

    Yi Tay, Mostafa Dehghani, Samira Abnar, Hyung Chung, William Fedus, Jinfeng Rao, Sharan Narang, Vinh Tran, Dani Yogatama, and Donald Metzler. 2023. Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?. InFindings of the Association for Computational ...

    2023 doi

  27. [35]

    Thinh Hung Truong, Timothy Baldwin, Karin Verspoor, and Trevor Cohn. 2023. Language Models Are Not Naysayers: An Analysis of Language Models on Negation Benchmarks. InProceedings of the 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023), Alexis Palmer and...

    2023 doi

  28. [36]

    Emile van Krieken, Pasquale Minervini, Edoardo Ponti, and Antonio Vergari. 2025. Neurosymbolic Diffusion Models. arXiv:2505.13138 [cs.LG] https://arxiv.org/abs/2505.13138

    2025

  29. [37]

    Ashish Vaswani, Google Brain, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need.Advances in neural information processing systemsNips (2017), 5998–6008

    2017

  30. [38]

    Marcell Vazquez-Chanlatte and Markus N. Rabe. 2018.mvcisback/py-aiger: v2.0.0. https://doi.org/10.5281/zenodo.1405781

    2018 doi

  31. [39]

    Yuxuan Wan, Wenxuan Wang, Yiliu Yang, Youliang Yuan, Jen-tse Huang, Pinjia He, Wenxiang Jiao, and Michael R. Lyu. 2024. A & B == B & A: Triggering Logical Reasoning Failures in Large Language Models. https://doi.org/10.48550/arXiv.2401.00757 arXiv:2401.00757 [cs]

    2024 doi arXiv

  32. [40]

    Sean Welleck, Peter West, Jize Cao, and Yejin Choi. 2022. Symbolic Brittleness in Sequence Models: On Systematic Generaliza- tion in Symbolic Mathematics. arXiv:2109.13986 [cs] http://arxiv.org/abs/2109.13986

    2022 arXiv

  33. [41]

    Jiacheng Ye, Jiahui Gao, Shansan Gong, Lin Zheng, Xin Jiang, Zhenguo Li, and Lingpeng Kong. 2025. Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning. InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24...

    2025

  34. [42]

    Emre Yolcu and Barnabás Póczos. [n. d.]. Learning Local Search Heuristics for Boolean Satisfiability. ([n. d.])

  35. [43]

    Honghua Zhang, Liunian Harold Li, Tao Meng, Kai-Wei Chang, and Guy Van den Broeck. 2022. On the Paradox of Learning to Reason from Data. arXiv:2205.11502 [cs] http://arxiv.org/abs/2205.11502 A The Dataset A.1 Imbalance in the dataset Preliminary inspection of the dataset showe...

    2022 arXiv

  36. [2022]

    Neural-Symbolic Learning and Reasoning: A Survey and Interpretation.Neuro-Symbolic Artificial Intelligence: The State of the Art342, 1 (2022), 327

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.