Pith. sign in

REVIEW 3 major objections 5 minor 91 references

Neuro-Symbolic Frameworks: Conceptual Characterization and Empirical Comparative Analysis

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Generic neurosymbolic frameworks can be characterized by five technical facets, and a four-task benchmark shows measurable speed-memory tradeoffs among DeepProbLog, Scallop, and DomiKnowS.

desk verdict A useful conceptual taxonomy of NeSy frameworks, but the efficiency table is too thin to support the claimed tradeoffs. read the letter →

arxiv 2509.07122 v1 pith:6ZH4HJ4Q submitted 2025-09-08 cs.AI cs.CLcs.SC

Sania Sinha , Tanawan Premsri , Danial Kamali , Parisa Kordjamshidi This is my paper · ORCID
classification cs.AIcs.CLcs.SC
keywords neurosymbolicAIframeworkcomparisonDeepProbLogScallopDomiKnowSdeclarativelearningdifferentiablereasoningLLM-assistedsymbolicknowledge
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Neurosymbolic AI aims to join neural learning with symbolic reasoning, but most work is algorithmic and task-specific rather than offering general-purpose tools. The paper's central claim is that the few generic frameworks that do exist can be understood through five facets: symbolic knowledge representation, neural modeling flexibility, model declaration, the interplay between symbolic and sub-symbolic systems, and use of large language models. It demonstrates the facets on DeepProbLog, Scallop, and DomiKnowS by implementing shared tasks (MNIST Sum, Shapes, Toy NER, Math Equation Inference, and a simple VQA) and measuring training time, inference time, and memory use. The empirical results show Scallop with the fastest inference, DeepProbLog training slightly faster in some settings, and DomiKnowS slower to train because it loads the whole domain graph into memory. A sympathetic reader would care because the facet characterization converts framework selection from a matter of taste into a structured engineering tradeoff and points to what the next generation of frameworks should fix.

What carries the argument

The carrying mechanism is the five-facet characterization summarized in Table 1 and then instantiated by shared tasks. Table 1 encodes each framework's symbolic language (ProbLog vs Datalog vs graph/FOL-like constraints), whether model declaration is flexible, which interplay algorithm is supported, computational-efficiency notes, and whether LLMs are used. The tasks act as a controlled stage on which the facets produce observable differences; Table 2 records per-sample training time, testing time, and memory for that purpose. The theoretical engine behind the efficiency differences is the choice of underlying reasoning formalism: exhaustive probabilistic inference over arithmetic circuits in DeepProbLog, top-k Datalog with provenance semirings in Scallop, and ILP-based constraint solving over an explicit domain graph in DomiKnowS.

What would settle it

Run the same tasks with the same neural backbone, optimizer, batch size, and hardware, using implementations maintained independently for each framework; if DeepProbLog's training-speed advantage and Scallop's inference-speed advantage disappear or reverse, the reported tradeoffs are implementation artifacts rather than properties of the frameworks themselves.

Watch

Extended reading notes

Core claim

The paper's own claim is that differences among generic neurosymbolic frameworks are not incidental implementation details but consequences of design choices across five facets. DeepProbLog builds on ProbLog's exhaustive probabilistic semantics with arithmetic circuits; Scallop relaxes those semantics with Datalog-style top-k reasoning and provenance semirings in a Rust implementation; DomiKnowS treats domain knowledge as a graph with logical constraints solved by integer linear programming, Lagrangian primal-dual training, or sampling losses. The tabulated results report that Scallop achieves the fastest testing times, DeepProbLog is slightly faster to train than Scallop in several settings, and DomiKnowS training is the slowest because of graph-loading overhead. The same facet lens shows that current frameworks each support only a narrow slice of possible symbolic-neural interplays, which the paper argues should be broadened in future designs.

Load-bearing premise

The claim depends on the four toy tasks and the authors' implementations being representative of each framework's general expressivity, and on wall-clock time and memory differences reflecting framework-level algorithmic properties rather than uneven engineering effort.

Editorial extensions

If this is right

  • If the facet characterization holds, framework selection becomes a requirements analysis: tasks needing exact probabilistic semantics fit DeepProbLog, tasks needing scalable differentiable Datalog reasoning fit Scallop, and tasks needing per-concept distant supervision fit DomiKnowS.
  • If the benchmark numbers generalize, a deployment that runs many inference queries should favor Scallop's speed, while a training-heavy research setting might tolerate DomiKnowS's slower training in exchange for concept-level losses.
  • If the facet lens is adopted, future framework papers should report symbolic representation language, model-declaration flexibility, supported interplay algorithms, LLM integration, and per-sample time and memory, making comparisons reproducible.
  • If the identified gaps are real, next-generation frameworks should support multiple types of symbolic-neural interplay in one system and use LLMs to generate or refine symbolic knowledge, lowering the hand-crafting barrier.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • I infer that the precise time and memory ratios should be treated as hypotheses rather than settled facts, because they come from the authors' own implementations; a controlled reimplementation with matched neural backbones could confirm or overturn the framework-level ordering.
  • I infer that the five-facet rubric could be applied to newer frameworks to predict integration effort before benchmarking, for example by scoring each facet and correlating it with measured user task-completion time.
  • I infer a testable claim about algorithm versus engineering: if Scallop's Datalog-style top-k optimizations are the true source of its speed, replacing its Rust runtime with a slower interpreter should preserve a smaller but still measurable advantage over DeepProbLog's exhaustive circuits.
  • I infer that the LLM facet could be quantified by measuring user time to specify a new task in natural language versus hand-crafted rules, which would test the claimed advantage of LLM-assisted model declaration.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a conceptual characterization of neuro-symbolic (NeSy) frameworks along five facets: symbolic knowledge representation, neural modeling flexibility, model declaration, the interplay between symbolic and sub-symbolic components, and the use of large language models. It illustrates the taxonomy with three frameworks (DeepProbLog, Scallop, DomiKnowS) and adds LEFT for one visual question answering task. The authors provide code-level examples for four tasks (MNIST Sum, Shapes, Toy NER, and Math Equation Inference) and report wall-clock time and memory measurements in Table 2. The stated contributions are the identification of the five facets, a cross-framework comparison, and a list of requirements and challenges for future NeSy frameworks.

Significance. If the characterization is accepted, it offers a useful organizing vocabulary for a fragmented field and a reasonable starting point for principled framework comparison. The paper is strongest in its qualitative sections: the five-facet schema is coherent, the code snippets in Figures 2 and 3 make framework differences concrete, and the public repository of examples is a practical resource. The empirical part is the weakest element: the current measurements do not support the claimed framework-level efficiency tradeoffs because accuracy is not reported, statistical power is very low, and implementation differences are uncontrolled. A redesigned empirical study, or a clear framing of the paper as a purely conceptual contribution with anecdotal illustrations, would make the contribution more defensible.

major comments (3)
  1. [Section 6, Table 2] The empirical comparison does not support the paper's central claim that the four tasks reveal measurable differences between frameworks. The text states that Table 2 compares efficiency 'on a single training/testing example,' yet per-sample wall-clock times in milliseconds can be dominated by one-off overheads such as library import, JIT compilation, and graph construction. No accuracy or loss is reported for any task, so it is impossible to determine whether the implementations actually learned the tasks; a fast-but-failed run would still enter the table. For the only entries with standard deviations (MNIST Sum training), DeepProbLog at 5.84 +/- 3.24 ms and Scallop at 6.50 +/- 2.35 ms overlap substantially, so the conclusion that 'DeepProbLog achieves slightly faster training performance than Scallop' is not statistically supported. The claims of fastest inference and memory efficiency in Section 6 and the tradeoff discussion in Section 9 therefore rest on an inadequate empirical basis.
  2. [Section 8, Table 2] The four tasks are all small, synthetic examples (two-digit MNIST Sum, synthetic Shapes, randomly generated Toy NER embeddings, and random-number Math Equation Inference), and all implementations are provided by the authors. The manuscript does not report hardware, training epochs, optimization hyperparameters, number of trials beyond five runs, or any convergence criteria, making the numbers non-reproducible. Because the authors are also the developers of DomiKnowS, differential familiarity with one framework is an uncontrolled confound. The text itself attributes DomiKnowS's slower training to 'the overhead of uploading the entire graph of data into memory' and DeepProbLog-Scallop differences to 'overhead unrelated to the core algorithmic complexity,' which concedes that the measurements reflect implementation details rather than framework-level algorithmic properties. Without independent implementations or at least a comparison using the official repositories and a detailed experimental protocol, the claimed efficiency tradeoffs are not established.
  3. [Section 3] The claim that 'Datalog can use top-k results and exploit database optimizations, making Scallop algorithmically more time-efficient than DeepProbLog' is presented as an established fact, but no algorithmic complexity analysis or reference is provided. This theoretical expectation is then used in Section 6 to interpret the empirical results, so it is a load-bearing part of the efficiency comparison. It should either be formally justified or explicitly hedged as a conjecture.
minor comments (5)
  1. [Table 1] The layout of Table 1 is difficult to read: the 'Eff' column appears to contain citations (e.g., 'Faghihi et al. (2024)') and check/cross marks are placed in ambiguous columns; the table should be reformatted so that each facet column is clearly aligned for every framework.
  2. [Abstract and Section 8] The abstract says the paper showcases three generic frameworks, but Section 8 also implements LEFT for the Simple VQA task; the paper should either present LEFT as a fourth showcased framework or clearly state that LEFT is an additional illustrative example only.
  3. [Section 7 and Table 1] Section 7 discusses DomiKnowS's use of LLMs (referencing Prompt2DeModel), but Table 1 marks the LLM column for DomiKnowS as '✗'; this inconsistency should be resolved.
  4. [Section 8.4] The Math Equation Inference task states that each list contains six real numbers, but the property descriptions use 'P8 i=0 xi > 0' and 'P8 i=0 |xi| > 0.5', which suggests a sum over nine elements; the indexing should be corrected to match the list length.
  5. [General] The paper would benefit from an explicit Limitations subsection that acknowledges the toy-scale tasks, the lack of independent implementations, and the potential for implementation artifacts in the timing measurements, as these are currently only implicit in the discussion.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's taxonomy and empirical comparisons are self-contained, and self-citations are contextual rather than load-bearing.

full rationale

Walking the paper's derivation chain, I find no step in which a claimed result is equivalent by construction to an input, fitted parameter, or self-referential assumption. The five-facet characterization in Section 2 and Table 1 is a conceptual taxonomy, not a derived prediction, and it is applied consistently across frameworks. The four task implementations in Section 8 are independent demonstrations with public code, and the efficiency figures in Table 2 are reported measurements rather than quantities implied by the characterization. The paper's own admission that timings were taken on a single training/testing example and that discrepancies may be due to overhead weakens the empirical claims, but that is a validity/statistical concern, not circularity. Self-citations to DomiKnowS and related declarative-learning work appear in Sections 2, 7, and 9 as contextual motivation for design preferences, not as a uniqueness theorem or as the source of a claimed result. No parameter is fitted and then renamed as a prediction, and no ansatz is imported solely through a self-citation chain. The empirical comparisons against DeepProbLog and Scallop provide external content that is falsifiable and not determined by the paper's own framework definition. Thus no circular step is present.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters are involved; the efficiency numbers are measurements rather than fitted values. The paper relies on framing assumptions about the sufficiency of its five facets, the representativeness of the four toy tasks, and the meaningfulness of wall-clock metrics, none of which are independently validated. No new entities are invented.

assumptions (3)
  • domain assumption The five selected facets (symbolic representation, neural modeling, model declaration, interplay, LLM usage) are sufficient to characterize NeSy frameworks.
    Section 2 introduces these facets as the comparison basis, but no validation shows they capture all dimensions of expressivity or usability for framework selection.
  • domain assumption The four example tasks cover the range of problems that generic NeSy frameworks are meant to solve.
    Section 8 presents MNIST Sum, Shapes VQA, Toy NER, and Math Equation Inference as the only empirical basis; all are small-scale and synthetic, so generalizability is assumed without evidence.
  • domain assumption Per-sample training time, testing time, and memory are meaningful proxies for framework efficiency and scalability.
    Table 2 contains only these metrics; the paper interprets them as efficiency comparisons in Section 6 without reporting accuracy, variance, or scaling behavior.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Neuro-Symbolic Frameworks: Conceptual Characterization and Empirical Comparative Analysis." pith.science (2026). https://pith.science/paper/6ZH4HJ4Q

@misc{pith2026250907122,
  author       = {Pith},
  title        = {Pith review of: Neuro-Symbolic Frameworks: Conceptual Characterization and Empirical Comparative Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6ZH4HJ4Q}},
  note         = {Machine review of arXiv:2509.07122}
}
read the original abstract

Neurosymbolic (NeSy) frameworks combine neural representations and learning with symbolic representations and reasoning. Combining the reasoning capacities, explainability, and interpretability of symbolic processing with the flexibility and power of neural computing allows us to solve complex problems with more reliability while being data-efficient. However, this recently growing topic poses a challenge to developers with its learning curve, lack of user-friendly tools, libraries, and unifying frameworks. In this paper, we characterize the technical facets of existing NeSy frameworks, such as the symbolic representation language, integration with neural models, and the underlying algorithms. A majority of the NeSy research focuses on algorithms instead of providing generic frameworks for declarative problem specification to leverage problem solving. To highlight the key aspects of Neurosymbolic modeling, we showcase three generic NeSy frameworks - \textit{DeepProbLog}, \textit{Scallop}, and \textit{DomiKnowS}. We identify the challenges within each facet that lay the foundation for identifying the expressivity of each framework in solving a variety of problems. Building on this foundation, we aim to spark transformative action and encourage the community to rethink this problem in novel ways.

Figures

Figures reproduced from arXiv: 2509.07122 by the authors.

Figure 1
Figure 1. An overview of the main components of a neurosymbolic framework. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Comparison of Symbolic Represen￾tation across frameworks. followed by ProbLog, its probabilistic ex￾tension. Finally, Scallop adopts a relational data model for symbolic knowledge repre￾sentation (Kolaitis and Vardi, 1990). Scal￾lop is built on top of the syntax and for￾mal semantics of Datalog and its proba￾bilistic extensions, relaxing the exact se￾mantics of ProbLog. It allows for the ex￾pression of common reason… view at source ↗
Figure 3
Figure 3. For neural integration, DomiKnowS utilizes sensors and readers for reading in data, while a learner connects to a network. DeepProbLog connects the neural network to the ProbLog file, requiring data handling to construct the terms and queries from the raw data. Scallop has an additional layer on top of the standard network that adds the symbolic context. expressions across frameworks, highlight￾ing differences in im… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Examples from the Shapes dataset for the question “Is there a red shape above a [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

91 extracted references · 29 canonical work pages

  1. [1]

    Foundations of Databases: The Logical Level

    Serge Abiteboul, Richard Hull, and Victor Vianu. Foundations of Databases: The Logical Level. Addison-Wesley Longman Publishing Co., Inc., USA, 1st edition, 1995. ISBN 0201537710

    1995

  2. [2]

    A survey on symbolic knowledge distillation of large language models

    Kamal Acharya, Alvaro Velasquez, and Houbing Herbert Song. A survey on symbolic knowledge distillation of large language models. IEEE Transactions on Artificial Intelligence, 5 0 (12): 0 5928--5948, 2024. doi:10.1109/TAI.2024.3428519

    work page arXiv 2024

  3. [3]

    Pyreason: Software for open world temporal logic, 2023

    Dyuman Aditya, Kaustuv Mukherji, Srikar Balasubramanian, Abhiraj Chaudhary, and Paulo Shakarian. Pyreason: Software for open world temporal logic, 2023. URL https://arxiv.org/abs/2302.13482

    arXiv 2023

  4. [4]

    Deep Learning Methods and Applications, pages 31--42

    Jamil Ahmad, Haleem Farman, and Zahoor Jan. Deep Learning Methods and Applications, pages 31--42. Springer Singapore, Singapore, 2019. ISBN 978-981-13-3459-7. doi:10.1007/978-981-13-3459-7_3. URL https://doi.org/10.1007/978-981-13-3459-7_3

    work page doi:10.1007/978-981-13-3459-7_3 2019

  5. [5]

    Pylon: A pytorch framework for learning with constraints

    Kareem Ahmed, Tao Li, Thy Ton, Quan Guo, Kai-Wei Chang, Parisa Kordjamshidi, Vivek Srikumar, Guy Van den Broeck, and Sameer Singh. Pylon: A pytorch framework for learning with constraints. In Douwe Kiela, Marco Ciccone, and Barbara Caputo, editors, Proceedings of the NeurIPS 2021 Competitions and Demonstrations Track, volume 176 of Proceedings of Machine ...

    2021

  6. [6]

    Luis M. Augusto. From symbols to knowledge systems: A. newell and h. a. S imon's contribution to symbolic AI . Journal of Knowledge Structures and Systems, 2 0 (1): 0 29--62, 2021

    2021

  7. [7]

    Logic tensor networks

    Samy Badreddine, Artur d'Avila Garcez , Luciano Serafini, and Michael Spranger. Logic tensor networks. Artificial Intelligence, 303: 0 103649, 2022. ISSN 0004-3702. doi:https://doi.org/10.1016/j.artint.2021.103649. URL https://www.sciencedirect.com/science/article/pii/S0004370221002009

    arXiv 2022

  8. [8]

    Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell

    Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT '21, page 610–623, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450383097. doi:10.1...

    arXiv 2021

Show all 91 references
  1. [9]

    Bikram Pratim Bhuyan, Amar Ramdane-Cherif, Ravi Tomar, and T. P. Singh. Neuro-symbolic artificial intelligence: a survey. Neural Computing and Applications, 36 0 (21): 0 12809--12844, July 2024. ISSN 1433-3058. doi:10.1007/s00521-024-09960-z. URL https://doi.org/10.1007/s00521...

    2024 doi

  2. [10]

    Thinking fast and slow in ai

    Grady Booch, Francesco Fabiano, Lior Horesh, Kiran Kate, Jonathan Lenchner, Nick Linck, Andreas Loreggia, Keerthiram Murgesan, Nicholas Mattei, Francesca Rossi, and Biplav Srivastava. Thinking fast and slow in ai. Proceedings of the AAAI Conference on Artificial Intelligence, ...

    2021 doi

  3. [11]

    Aggarwal

    Djallel Bouneffouf and Charu C. Aggarwal. Survey on applications of neurosymbolic artificial intelligence, 2022. URL https://arxiv.org/abs/2209.12618

    2022 arXiv

  4. [12]

    Applications of inductive logic programming

    Ivan Bratko and Stephen Muggleton. Applications of inductive logic programming. Commun. ACM, 38 0 (11): 0 65–70, November 1995. ISSN 0001-0782. doi:10.1145/219717.219771. URL https://doi.org/10.1145/219717.219771

    1995

  5. [13]

    Burattini, A

    E. Burattini, A. de Francesco, and M. De Gregorio. Nsl: a neuro-symbolic language for monotonic and non-monotonic logical inferences. In VII Brazilian Symposium on Neural Networks, 2002. SBRN 2002. Proceedings., pages 256--261, 2002. doi:10.1109/SBRN.2002.1181487

    2002 arXiv

  6. [14]

    Programming in PROLOG

    William F Clocksin and Christopher S Mellish. Programming in PROLOG. Springer Science & Business Media, 2003

    2003

  7. [15]

    Tensorlog: Deep learning meets probabilistic dbs

    William W Cohen, Fan Yang, and Kathryn Rivard Mazaitis. Tensorlog: Deep learning meets probabilistic dbs. arXiv preprint arXiv:1707.05390, 2017

    2017 arXiv

  8. [16]

    Inductive logic programming at 30: A new introduction

    Andrew Cropper and Sebastijan Duman c i\' c . Inductive logic programming at 30: A new introduction. J. Artif. Int. Res., 74, September 2022. ISSN 1076-9757. doi:10.1613/jair.1.13507. URL https://doi.org/10.1613/jair.1.13507

    2022 doi

  9. [17]

    Sdd: A new canonical representation of propositional knowledge bases

    Adnan Darwiche. Sdd: A new canonical representation of propositional knowledge bases. In IJCAI Proceedings-International Joint Conference on Artificial Intelligence, volume 22, page 819, 2011

    2011

  10. [18]

    Problog: a probabilistic prolog and its application in link discovery

    Luc De Raedt, Angelika Kimmig, and Hannu Toivonen. Problog: a probabilistic prolog and its application in link discovery. In Proceedings of the 20th International Joint Conference on Artifical Intelligence, IJCAI'07, page 2468–2473, San Francisco, CA, USA, 2007. Morgan Kaufman...

    2007

  11. [19]

    The deeplog neurosymbolic machine, 2025

    Vincent Derkinderen, Robin Manhaeve, Rik Adriaensen, Lucas Van Praet, Lennert De Smet, Giuseppe Marra, and Luc De Raedt. The deeplog neurosymbolic machine, 2025. URL https://arxiv.org/abs/2508.13697

    2025

  12. [20]

    Parameter estimation for probabilistic finite-state transducers

    Jason Eisner. Parameter estimation for probabilistic finite-state transducers. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 1--8, 2002

    2002

  13. [21]

    Plan- SOFAI : A neuro-symbolic planning architecture

    Francesco Fabiano, Vishal Pallagani, Marianna Bergamaschi Ganapini, Lior Horesh, Andrea Loreggia, Keerthiram Murugesan, Francesca Rossi, and Biplav Srivastava. Plan- SOFAI : A neuro-symbolic planning architecture. In Neuro-Symbolic Learning and Reasoning in the era of Large La...

    2023

  14. [22]

    Gluecons: A generic benchmark for learning under constraints, 2023

    Hossein Rajaby Faghihi, Aliakbar Nafar, Chen Zheng, Roshanak Mirzaee, Yue Zhang, Andrzej Uszok, Alexander Wan, Tanawan Premsri, Dan Roth, and Parisa Kordjamshidi. Gluecons: A generic benchmark for learning under constraints, 2023. URL https://arxiv.org/abs/2302.10914

    2023 arXiv

  15. [23]

    Prompt2demodel: Declarative neuro-symbolic modeling with natural language, 2024

    Hossein Rajaby Faghihi, Aliakbar Nafar, Andrzej Uszok, Hamid Karimian, and Parisa Kordjamshidi. Prompt2demodel: Declarative neuro-symbolic modeling with natural language, 2024. URL https://arxiv.org/abs/2407.20513

    2024 arXiv

  16. [24]

    Large language models are neurosymbolic reasoners

    Chuyu Fang and Song-Chun Yu. Large language models are neurosymbolic reasoners. arXiv preprint arXiv:2401.09334, 2024. URL https://arxiv.org/html/2401.09334v1

    2024 arXiv

  17. [25]

    Learning from entailment: An application to propositional horn sentences

    Michael Frazier and Leonard Pitt. Learning from entailment: An application to propositional horn sentences. In Proceedings of the Tenth International Conference on International Conference on Machine Learning, pages 120--127, 1993

    1993

  18. [26]

    Ccn+: A neuro-symbolic framework for deep learning with requirements

    Eleonora Giunchiglia, Alex Tatomir, Mihaela Cătălina Stoian, and Thomas Lukasiewicz. Ccn+: A neuro-symbolic framework for deep learning with requirements. International Journal of Approximate Reasoning, 171: 0 109124, 2024. ISSN 0888-613X. doi:https://doi.org/10.1016/j.ijar.20...

    2024

  19. [27]

    Green, Grigoris Karvounarakis, and Val Tannen

    Todd J. Green, Grigoris Karvounarakis, and Val Tannen. Provenance semirings. In Proceedings of the Twenty-Sixth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, PODS '07, page 31–40, New York, NY, USA, 2007. Association for Computing Machinery. ISBN 978159...

    2007

  20. [28]

    Inference-masked loss for deep structured output learning

    Quan Guo, Hossein Rajaby Faghihi, Yue Zhang, Andrzej Uszok, and Parisa Kordjamshidi. Inference-masked loss for deep structured output learning. In Christian Bessiere, editor, Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20 , ...

    2020 doi

  21. [29]

    Gurobi Optimizer Reference Manual , 2024

    Gurobi Optimization, LLC . Gurobi Optimizer Reference Manual , 2024. URL https://www.gurobi.com

    2024

  22. [30]

    Rule-based systems

    Frederick Hayes-Roth. Rule-based systems. Communications of the ACM, 28 0 (9): 0 921--932, 1985

    1985

  23. [31]

    Neuro-symbolic artificial intelligence: The state of the art

    Pascal Hitzler and Md Kamruzzaman Sarker. Neuro-symbolic artificial intelligence: The state of the art. 2022

    2022

  24. [32]

    A study on neuro-symbolic artificial intelligence: Healthcare perspectives, 2025

    Delower Hossain and Jake Y Chen. A study on neuro-symbolic artificial intelligence: Healthcare perspectives, 2025. URL https://arxiv.org/abs/2503.18213

    2025 arXiv

  25. [33]

    Tenenbaum, and Jiajun Wu

    Joy Hsu, Jiayuan Mao, Joshua B. Tenenbaum, and Jiajun Wu. What's left? concept grounding with logic-enhanced foundation models, 2023. URL https://arxiv.org/abs/2310.16035

    2023 arXiv

  26. [34]

    Scallop: From probabilistic deductive databases to scalable differentiable reasoning

    Jiani Huang, Ziyang Li, Binghong Chen, Karan Samel, Mayur Naik, Le Song, and Xujie Si. Scallop: From probabilistic deductive databases to scalable differentiable reasoning. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neur...

    2021

  27. [35]

    Leveraging large language models to generate answer set programs

    Adam Ishay, Zhun Yang, and Joohyung Lee. Leveraging large language models to generate answer set programs. In Pierre Marquis, Tran Cao Son, and Gabriele Kern-Isberner, editors, Proceedings of the 20th International Conference on Principles of Knowledge Representation and Reaso...

    2023 doi

  28. [36]

    Neurosymbolic approaches in ai design – an overview

    Prashani Jayasingha, Bogdan Iancu, and Johan Lilius. Neurosymbolic approaches in ai design – an overview. In 2025 IEEE Symposium on Trustworthy, Explainable and Responsible Computational Intelligence (CITREx Companion), pages 1--5, 2025. doi:10.1109/CITRExCompanion65208.2025.10981497

    2025

  29. [37]

    Lawrence Zitnick, and Ross Girshick

    Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C. Lawrence Zitnick, and Ross Girshick. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogniti...

    2017

  30. [38]

    Thinking, fast and slow

    Daniel Kahneman. Thinking, fast and slow. macmillan, 2011

    2011

  31. [39]

    Barezi, and Parisa Kordjamshidi

    Danial Kamali, Elham J. Barezi, and Parisa Kordjamshidi. Nesycoco: A neuro-symbolic concept composer for compositional generalization. Proceedings of the AAAI Conference on Artificial Intelligence, 39 0 (4): 0 4184--4193, Apr. 2025. doi:10.1609/aaai.v39i4.32439. URL https://oj...

    2025 doi

  32. [40]

    The third ai summer: Aaai robert s

    Henry Kautz. The third ai summer: Aaai robert s. engelmore memorial lecture. AI Magazine, 43 0 (1): 0 105--125, Mar. 2022. doi:10.1002/aaai.12036. URL https://ojs.aaai.org/aimagazine/index.php/aimagazine/article/view/19122

    2022 doi

  33. [41]

    An algebraic prolog for reasoning about possible worlds

    Angelika Kimmig, Guy Van den Broeck, and Luc De Raedt. An algebraic prolog for reasoning about possible worlds. Proceedings of the AAAI Conference on Artificial Intelligence, 25 0 (1): 0 209--214, Aug. 2011. doi:10.1609/aaai.v25i1.7852. URL https://ojs.aaai.org/index.php/AAAI/...

    2011 doi

  34. [42]

    Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything, 2023. URL https://arxiv.org/abs/2304.02643

    2023 arXiv

  35. [43]

    Kolaitis and Moshe Y

    Phokion G. Kolaitis and Moshe Y. Vardi. On the expressive power of datalog: tools and a case study. In Proceedings of the Ninth ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, PODS '90, page 61–71, New York, NY, USA, 1990. Association for Computing Machin...

    1990

  36. [44]

    Saul: Towards declarative learning based programming

    Parisa Kordjamshidi, Dan Roth, and Hao Wu. Saul: Towards declarative learning based programming. volume 2015, 12 2015

    2015

  37. [45]

    Better call S aul: Flexible programming for learning and inference in NLP

    Parisa Kordjamshidi, Daniel Khashabi, Christos Christodoulopoulos, Bhargav Mangipudi, Sameer Singh, and Dan Roth. Better call S aul: Flexible programming for learning and inference in NLP . In Yuji Matsumoto and Rashmi Prasad, editors, Proceedings of COLING 2016, the 26th Inte...

    2016

  38. [46]

    Declarative learning-based programming as an interface to ai systems

    Parisa Kordjamshidi, Dan Roth, and Kristian Kersting. Declarative learning-based programming as an interface to ai systems. Frontiers in Artificial Intelligence, 5, 2019. URL https://api.semanticscholar.org/CorpusID:195069123

    2019

  39. [47]

    Declarative learning-based programming as an interface to ai systems

    Parisa Kordjamshidi, Dan Roth, and Kristian Kersting. Declarative learning-based programming as an interface to ai systems. Frontiers in Artificial Intelligence, Volume 5 - 2022, 2022. ISSN 2624-8212. doi:10.3389/frai.2022.755361. URL https://www.frontiersin.org/journals/artif...

    2022

  40. [48]

    Lamb, Artur Garcez, Marco Gori, Marcelo Prates, Pedro Avelar, and Moshe Vardi

    Luis C. Lamb, Artur Garcez, Marco Gori, Marcelo Prates, Pedro Avelar, and Moshe Vardi. Graph neural networks meet neural-symbolic computing: A survey and perspective, 2021. URL https://arxiv.org/abs/2003.00330

    2021 arXiv

  41. [49]

    Deep learning for symbolic mathematics

    Guillaume Lample and François Charton. Deep learning for symbolic mathematics. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=S1eZYeHFDS

    2020

  42. [50]

    Lecun, L

    Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86 0 (11): 0 2278--2324, 1998. doi:10.1109/5.726791

    1998 doi

  43. [51]

    Deep learning

    Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521 0 (7553): 0 436--444, 2015. doi:10.1038/nature14539. URL https://doi.org/10.1038/nature14539

    2015 doi

  44. [52]

    Trustworthy ai: From principles to practices

    Bo Li, Peng Qi, Bo Liu, Shuai Di, Jingen Liu, Jiquan Pei, Jinfeng Yi, and Bowen Zhou. Trustworthy ai: From principles to practices. ACM Comput. Surv., 55 0 (9), January 2023 a . ISSN 0360-0300. doi:10.1145/3555803. URL https://doi.org/10.1145/3555803

    2023 doi

  45. [53]

    Scallop: A language for neurosymbolic programming

    Ziyang Li, Jiani Huang, and Mayur Naik. Scallop: A language for neurosymbolic programming. Proc. ACM Program. Lang., 7 0 (PLDI), June 2023 b . doi:10.1145/3591280. URL https://doi.org/10.1145/3591280

    2023 doi

  46. [54]

    Relational programming with foundational models

    Ziyang Li, Jiani Huang, Jason Liu, Felix Zhu, Eric Zhao, William Dodds, Neelay Velingker, Rajeev Alur, and Mayur Naik. Relational programming with foundational models. Proceedings of the AAAI Conference on Artificial Intelligence, 38 0 (9): 0 10635–10644, March 2024. ISSN 2159...

    2024 doi

  47. [55]

    Satyrus: A sat-based neuro-symbolic architecture for constraint processing

    Priscila Lima, Mariela Morveli-Espinoza, Glaucia K E Pereira, and Felipe França. Satyrus: A sat-based neuro-symbolic architecture for constraint processing. pages 137--142, 01 2005. doi:10.1109/ICHIS.2005.97

    2005 doi

  48. [56]

    Runtao Liu, Chenxi Liu, Yutong Bai, and Alan L. Yuille. Clevr-ref+: Diagnosing visual reasoning with referring expressions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019

    2019

  49. [57]

    Surveying neuro-symbolic approaches for reliable artificial intelligence of things

    Zhen Lu, Imran Afridi, Hong Jin Kang, Ivan Ruchkin, and Xi Zheng. Surveying neuro-symbolic approaches for reliable artificial intelligence of things. Journal of Reliable Intelligent Environments, 10 0 (3): 0 257--279, 2024. doi:10.1007/s40860-024-00231-1. URL https://doi.org/1...

    2024 doi

  50. [58]

    Neural probabilistic logic programming in deepproblog

    Robin Manhaeve, Sebastijan Dumančić, Angelika Kimmig, Thomas Demeester, and Luc De Raedt . Neural probabilistic logic programming in deepproblog. Artificial Intelligence, 298: 0 103504, 2021. ISSN 0004-3702. doi:https://doi.org/10.1016/j.artint.2021.103504. URL https://www.sci...

    2021

  51. [59]

    Tenenbaum, and Jiajun Wu

    Jiayuan Mao, Chuang Gan, Pushmeet Kohli, Joshua B. Tenenbaum, and Jiajun Wu. The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision, 2019. URL https://arxiv.org/abs/1904.12584

    2019 arXiv

  52. [60]

    Simple open-vocabulary object detection with vision transformers, 2022

    Matthias Minderer, Alexey Gritsenko, Austin Stone, Maxim Neumann, Dirk Weissenborn, Alexey Dosovitskiy, Aravindh Mahendran, Anurag Arnab, Mostafa Dehghani, Zhuoran Shen, Xiao Wang, Xiaohua Zhai, Thomas Kipf, and Neil Houlsby. Simple open-vocabulary object detection with vision...

    2022 arXiv

  53. [61]

    Disentangling extraction and reasoning in multi-hop spatial reasoning

    Roshanak Mirzaee and Parisa Kordjamshidi. Disentangling extraction and reasoning in multi-hop spatial reasoning. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 3379--3397, Singapore, December ...

    2023 doi

  54. [62]

    A primal dual formulation for deep learning with constraints

    Yatin Nandwani, Abhishek Pathak, Mausam, and Parag Singla. A primal dual formulation for deep learning with constraints. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d Alch\' e -Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume...

    2019

  55. [63]

    Physical symbol systems

    Allen Newell. Physical symbol systems. Cognitive Science, 4 0 (2): 0 135--183, 1980. ISSN 0364-0213. doi:https://doi.org/10.1016/S0364-0213(80)80015-2. URL https://www.sciencedirect.com/science/article/pii/S0364021380800152

    1980 doi

  56. [64]

    Subrahmanian

    Raymond Ng and V.S. Subrahmanian. Probabilistic logic programming. Information and Computation, 101 0 (2): 0 150--201, 1992. ISSN 0890-5401. doi:https://doi.org/10.1016/0890-5401(92)90061-J. URL https://www.sciencedirect.com/science/article/pii/089054019290061J

    1992

  57. [65]

    What is inductive logic programming? Springer, 1997

    Shan-Hwei Nienhuys-Cheng and Roland de Wolf. What is inductive logic programming? Springer, 1997

    1997

  58. [66]

    Gpt-4 technical report, 2024

    OpenAI, Josh Achiam, and Steven Adler et al. Gpt-4 technical report, 2024. URL https://arxiv.org/abs/2303.08774

    2024 arXiv

  59. [67]

    Logic- LM : Empowering large language models with symbolic solvers for faithful logical reasoning

    Liangming Pan, Alon Albalak, Xinyi Wang, and William Wang. Logic- LM : Empowering large language models with symbolic solvers for faithful logical reasoning. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP...

    2023 doi

  60. [68]

    Logic-lm: Empowering large language models with symbolic solvers for faithful logical reasoning, 2023 b

    Liangming Pan, Alon Albalak, Xinyi Wang, and William Yang Wang. Logic-lm: Empowering large language models with symbolic solvers for faithful logical reasoning, 2023 b . URL https://arxiv.org/abs/2305.12295

    2023 arXiv

  61. [69]

    Pytorch: An imperative style, high-performance deep learning library, 2019

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fan...

    2019 arXiv

  62. [70]

    Miller, and Sebastian Riedel

    Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H. Miller, and Sebastian Riedel. Language models as knowledge bases?, 2019. URL https://arxiv.org/abs/1909.01066

    2019 arXiv

  63. [71]

    Learning transferable visual models from natural language supervision, 2021

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. URL https://ar...

    2021 arXiv

  64. [72]

    D omi K now S : A library for integration of symbolic domain knowledge in deep learning

    Hossein Rajaby Faghihi, Quan Guo, Andrzej Uszok, Aliakbar Nafar, and Parisa Kordjamshidi. D omi K now S : A library for integration of symbolic domain knowledge in deep learning. In Heike Adel and Shuming Shi, editors, Proceedings of the 2021 Conference on Empirical Methods in...

    2021 doi

  65. [73]

    Learning rules comparison in neuro-symbolicintegration

    Saratha Sathasivam. Learning rules comparison in neuro-symbolicintegration. International Journal of Applied Physics and Mathematics, 1 0 (2): 0 129, 2011

    2011

  66. [74]

    Learning and reasoning with logic tensor networks

    Luciano Serafini and Artur S d’Avila Garcez. Learning and reasoning with logic tensor networks. In Conference of the italian association for artificial intelligence, pages 334--348. Springer, 2016

    2016

  67. [75]

    Arithmetic circuits: A survey of recent results and open questions

    Amir Shpilka, Amir Yehudayoff, et al. Arithmetic circuits: A survey of recent results and open questions. Foundations and Trends in Theoretical Computer Science , 5 0 (3--4): 0 207--388, 2010

    2010

  68. [76]

    David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madele...

    2016 doi

  69. [77]

    Basic reasoning with tensor product representations

    Paul Smolensky, Moontae Lee, Xiaodong He, Wen-tau Yih, Jianfeng Gao, and li Deng. Basic reasoning with tensor product representations. 01 2016. doi:10.48550/arXiv.1601.02745

    2016 doi arXiv

  70. [78]

    Neuro-symbolic program search for autonomous driving decision module design

    Jiankai Sun, Hao Sun, Tian Han, and Bolei Zhou. Neuro-symbolic program search for autonomous driving decision module design. In Jens Kober, Fabio Ramos, and Claire Tomlin, editors, Proceedings of the 2020 Conference on Robot Learning, volume 155 of Proceedings of Machine Learn...

    2020

  71. [79]

    Tjong Kim Sang and Fien De Meulder

    Erik F. Tjong Kim Sang and Fien De Meulder. Introduction to the C o NLL -2003 shared task: Language-independent named entity recognition. In Proceedings of the Seventh Conference on Natural Language Learning at HLT - NAACL 2003 , pages 142--147, 2003. URL https://aclanthology....

    2003

  72. [80]

    Llama 2: Open foundation and fine-tuned chat models, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, W...

    2023 arXiv

  73. [81]

    Constraint satisfaction using constraint logic programming

    Pascal Van Hentenryck , Helmut Simonis, and Mehmet Dincbas. Constraint satisfaction using constraint logic programming. Artificial Intelligence, 58 0 (1): 0 113--159, 1992. ISSN 0004-3702. doi:https://doi.org/10.1016/0004-3702(92)90006-J. URL https://www.sciencedirect.com/scie...

    1992

  74. [82]

    Eliza—a computer program for the study of natural language communication between man and machine

    Joseph Weizenbaum. Eliza—a computer program for the study of natural language communication between man and machine. Commun. ACM, 9 0 (1): 0 36–45, January 1966. ISSN 0001-0782. doi:10.1145/365153.365168. URL https://doi.org/10.1145/365153.365168

    1966

  75. [83]

    Lew, Noah D

    Lionel Wong, Gabriel Grand, Alexander K. Lew, Noah D. Goodman, Vikash K. Mansinghka, Jacob Andreas, and Joshua B. Tenenbaum. From word models to world models: Translating from natural language to the probabilistic language of thought, 2023. URL https://arxiv.org/abs/2306.12672

    2023 arXiv

  76. [84]

    Symbol- LLM : Towards foundational symbol-centric interface for large language models

    Fangzhi Xu, Zhiyong Wu, Qiushi Sun, Siyu Ren, Fei Yuan, Shuai Yuan, Qika Lin, Yu Qiao, and Jun Liu. Symbol- LLM : Towards foundational symbol-centric interface for large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual ...

    2024 doi

  77. [85]

    A semantic loss function for deep learning with symbolic knowledge

    Jingyi Xu, Zilu Zhang, Tal Friedman, Yitao Liang, and Guy Van den Broeck. A semantic loss function for deep learning with symbolic knowledge. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedi...

    2018

  78. [86]

    Faithful logical reasoning via symbolic chain-of-thought

    Xinyu Xu, Pan Wang, Shusen Dong, Ningyu Wu, Yue Zhang, and Baobao Chang. Faithful logical reasoning via symbolic chain-of-thought. arXiv preprint arXiv:2405.18357, 2024 b . URL https://arxiv.org/abs/2405.18357

    2024 arXiv

  79. [87]

    Arithmetic reasoning with LLM : P rolog generation & permutation

    Xiaocheng Yang, Bingsen Chen, and Yik-Cheung Tam. Arithmetic reasoning with LLM : P rolog generation & permutation. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Li...

    2024 doi

  80. [88]

    Exploring large language models for knowledge graph completion, 2025

    Liang Yao, Jiazhen Peng, Chengsheng Mao, and Yuan Luo. Exploring large language models for knowledge graph completion, 2025. URL https://arxiv.org/abs/2308.13916

    2025 arXiv

  81. [89]

    Tenenbaum

    Kexin Yi, Jiajun Wu, Chuang Gan, Antonio Torralba, Pushmeet Kohli, and Joshua B. Tenenbaum. Neural-symbolic vqa: Disentangling reasoning from vision and language understanding. In Advances in Neural Information Processing Systems, pages 1039--1050, 2018

    2018

  82. [90]

    Improved logical reasoning of language models via differentiable symbolic programming, 2023

    Hanlin Zhang, Jiani Huang, Ziyang Li, Mayur Naik, and Eric Xing. Improved logical reasoning of language models via differentiable symbolic programming, 2023. URL https://arxiv.org/abs/2305.03742

    2023 arXiv

  83. [91]

    Danna Zheng, Mirella Lapata, and Jeff Z. Pan. How reliable are llms as knowledge bases? re-thinking facutality and consistency, 2024. URL https://arxiv.org/abs/2407.13578

    2024 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.