Symbolic Knowledge Extraction and Injection with Sub-symbolic Predictors: A Systematic Literature Review

Andrea Agiollo; Andrea Omicini; Federico Sabbatini; Giovanni Ciatto; Matteo Magnini

REVIEW 2 major objections 5 minor 298 references

Symbolic Knowledge Extraction and Injection with Sub-symbolic Predictors: A Systematic Literature Review

T0 review · 2 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This systematic review claims that the many scattered techniques for symbolic knowledge extraction (SKE) and symbolic knowledge injection (SKI) can be captured by two general meta-models and two taxonomies.

desk verdict A genuinely useful joint SKE/SKI survey with transparent appendices, but the SKI taxonomy has a definitional inconsistency that a referee should press on. read the letter →

arxiv 2501.14836 v1 pith:OJAMLMRF submitted 2025-01-23 cs.AI cs.LGcs.LO

Giovanni Ciatto , Federico Sabbatini , Andrea Agiollo , Matteo Magnini , Andrea Omicini This is my paper · ORCID

classification cs.AIcs.LGcs.LO

keywords symbolicknowledgeextractioninjectionexplainableartificialintelligencetaxonomyneuralnetworksgraphsneuro-symboliccomputingsystematicliteraturereview

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This systematic review claims that the many scattered techniques for symbolic knowledge extraction (SKE) — turning a trained opaque predictor like a neural network into human-readable rules — and symbolic knowledge injection (SKI) — constraining a predictor with human-supplied knowledge — can be captured by two general meta-models and two taxonomies. The authors analyzed 132 SKE and 117 SKI methods from four decades of literature and organized them by what they accept as input, what they produce, how they operate, and why they are used. If the taxonomies hold, a data scientist can use them to select the right method for a task, a researcher can see where the field is empty, and a developer gets a map of which methods have runnable software. The review also frames SKE and SKI as two halves of an XAI story: extraction explains an opaque model, injection bypasses the need for transparency by making behavior predictable.

What carries the argument

The load-bearing machinery is the pair of meta-models plus the dimensions used to classify methods. The SKE dimensions are translucency (pedagogical methods treat the predictor as an oracle; decompositional methods inspect internal parameters), task (classification, regression), input data (binary, discrete, continuous, images, text, graphs), and output knowledge shape (rule lists, decision trees or tables, knowledge graphs) and expressiveness (propositional, fuzzy, oblique, M-of-N, triplets). The SKI dimensions are input formalism (first-order logic and its subsets, knowledge graphs, propositional logic, expert knowledge), injection strategy (predictor structuring, knowledge embedding, guided learning), target predictor (mostly neural-network families, plus Markov chains and kernel machines), and purpose (manipulating symbolic knowledge or enriching learning). These dimensions let each surveyed method be slotted into a cell, which is what turns a scattered literature into a navigable map.

What would settle it

Locate a published SKE method that extracts symbolic knowledge from an unsupervised or reinforcement-learning predictor, or an SKI method that injects knowledge into a predictor family outside neural networks, kernel machines, and Markov chains; either would fall outside the taxonomies as presented and would overturn the claim that the taxonomies cover all current SKE/SKI methods.

Watch

Extended reading notes

Core claim

The paper's central claim is that SKE and SKI are unified fields describable by explicit, general meta-models and taxonomies, not just labels for a miscellany of techniques. An SKE meta-model is any procedure that takes a trained sub-symbolic predictor and outputs symbolic knowledge reflecting the predictor's behavior with high fidelity; an SKI meta-model is any procedure that makes a predictor's inferences computed as a function of, or consistent with, given symbolic knowledge. From these definitions the authors induce bottom-up taxonomies: SKE methods sort by translucency (pedagogical vs decompositional), targeted AI task, input data type, and output knowledge shape and expressiveness; SKI methods sort by input knowledge formalism, strategy (structuring, embedding, guided learning), target predictor type, and purpose (symbolic manipulation vs learning support). The paper claims this is the only systematic treatment covering both activities and the largest collection to date (249 methods).

Load-bearing premise

The keyword-based search, limited to the first two pages of results from selected search engines and extended only by snowballing from 11 secondary works, is assumed to capture a representative sample of the SKE and SKI literature; if many works using different naming conventions were missed, the completeness of the taxonomies would be undermined.

Editorial extensions

If this is right

A practitioner can choose an SKE or SKI method by locating their predictor type, data, and task in the taxonomy, then filter by software availability.
The taxonomy exposes clear gaps: SKE is confined to supervised tasks, and SKI is dominated by knowledge graphs and propositional logic, so first-order logic with recursion and unsupervised or reinforcement learning settings are open research territory.
The train-extract-fix-inject (TEFI) loop makes it possible to debug a neural predictor: extract rules, spot wrong ones, fix them, and inject the corrections back.
Symbolic knowledge can act as a lingua franca among heterogeneous hybrid agents, letting them exchange and improve behavioral knowledge through SKE and SKI.
Only about 35 percent of the surveyed methods have runnable software, indicating where implementation effort is most needed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The taxonomy implicitly sets a research agenda: one could test its predictive power by taking a newly published method and checking whether its properties are fully determined by the proposed dimensions, or whether a new dimension is needed.
The authors' notions of fidelity (SKE) and consistency (SKI) are qualitative; formalizing them as measurable metrics would let the taxonomy support quantitative method comparison.
As large language models and other generative predictors become dominant sub-symbolic systems, these definitions and taxonomies would likely need new categories such as extraction from or injection into generative models, providing a natural test of the taxonomy's durability.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Quick take: this is a genuine synthesis—the first systematic review I know that treats SKE and SKI under one set of meta-models and taxonomies. The appendix tables alone are a useful contribution: 249 methods, classified by translucency, input/output types, expressiveness, and software availability. The counts check out against the tables, and the authors are transparent about their search protocol. The self-citations to [42] and [60,61] are backdrop, not load-bearing evidence.

The main soft spot is real: Section 4.2.1 introduces an 'expert knowledge' input category for SKI defined as 'human- (but not necessarily machine-) interpretable'—but Section 3.1.2 defines SKI as requiring input knowledge that is machine-interpretable, explicitly excluding free text or natural language. That is a direct internal contradiction. The category is populated (several entries in Table C use 'E'), so it is not a stray remark; it affects the SKI taxonomy's claim to faithfully generalize the concept. A referee should ask the authors to either adjust the SKI definition to admit knowledge that is reified via data generation, or delete the category and reclassify those methods. Neither is a huge job, but leaving it as-is undercuts the paper's precision.

The sampling limitation—first two pages of results per search engine—is acknowledged in Section 5.5. I agree it is a minor weakness, not a fatal one; this is a taxonomy intended to be updated, and the snowballing from 11 secondary works partially compensates. The absence of an inter-rater reliability check on the qualitative classification is worth a note, but at this scale it is a common limitation in SLRs.

Overall, the paper is a solid organizing contribution. It should go to review. The inconsistency in the SKI input taxonomy is the thing I would press on; fix that and this becomes a reference I'd happily point people to.

Referee Report

2 major / 5 minor

Summary. This paper presents a systematic literature review of symbolic knowledge extraction (SKE) and symbolic knowledge injection (SKI) methods for sub-symbolic predictors. The authors define SKE and SKI from an XAI perspective, conduct keyword-based searches on five bibliographic engines with snowballing from eleven secondary works, and classify 249 primary works (132 SKE, 117 SKI) along dimensions such as translucency, input data type, output knowledge shape and expressiveness for SKE, and input knowledge type, strategy, predictor type, and purpose for SKI. They propose two general meta-models and taxonomies, report the availability of runnable software implementations (87 of 249 methods), and discuss challenges and opportunities such as the train-extract-fix-inject loop.

Significance. If the proposed taxonomies are valid, the paper would be a useful reference for practitioners selecting SKE/SKI methods and for researchers identifying gaps in the field. The paper's strengths include a transparent search protocol, explicit inclusion criteria, and detailed appendices (Tables B and C) that enumerate every surveyed method with its classification and software URL, making the main statistics checkable. The authors also acknowledge terminology-related coverage limits in Section 5.5. However, the SKI taxonomy contains an internal contradiction with the paper's own definition of SKI: the 'expert knowledge' input category in Section 4.2.1 violates the machine-interpretability requirement stated in Section 3.1.2. Because this affects the empirical support for the SKI meta-model and the statistics derived from the 117-method corpus, the manuscript needs revision before the central claim can be fully accepted.

major comments (2)

[Section 3.1.2 and Section 4.2.1 / Figure 8 / Table C] Section 3.1.2 defines SKI as requiring machine-interpretable input knowledge, explicitly stating that 'some formal language ... should be employed ... while free text or natural language should be avoided.' Section 4.2.1, however, introduces an input-knowledge category called 'expert knowledge,' defined as 'any piece of human- (but not necessarily machine-) interpretable knowledge' and including knowledge 'only accessible to human beings' that requires data generation to reify it. Table C and Figure 8 include this category (with input type 'E'), so several surveyed methods are classified as SKI even though, by the paper's own Section 3.1.2 criterion, they do not satisfy the definition of SKI. This undermines the claim that the SKI taxonomy faithfully generalizes the concept of SKI and affects the distributional statistics and gap analysis derived from the 117-method corpus. I recommend either (a) revising the Section 3.1.2 definition to admit human-only-interpretable knowledge when it is reified by an explicit data-generation step, or (b) removing the 'expert knowledge' category and reclassifying the affected methods, with the consequent changes to Figures 8 and 14 and the discussion in Section 5.2.
[Section 3.2] The search protocol limits results to the first two pages of each query/engine pair and performs inclusion/exclusion without a second coder or inter-rater reliability check. Because the two taxonomies are induced bottom-up from the resulting corpus, the representativeness of this sample directly affects the completeness of the taxonomy dimensions and the quantitative gap analysis in Sections 4 and 5.3. I am not asking for an exhaustive search, but the authors should either provide a saturation or sensitivity analysis, or state more cautiously that the taxonomies are provisional with respect to search-engine ranking; the current text in Section 5.5 only acknowledges naming-convention misses.

minor comments (5)

[Section 3.1.1] The statement that 'symbolic' is intended here as 'a synonym of intelligible (for the human being),' making bare human-readable text an admissible SKE output, conflicts with the Introduction's definition of symbolic as intelligible for both humans and computers; please align these definitions.
[Section 3.2] Reporting a PRISMA-style flow diagram with the number of records retrieved, screened, excluded, and deduplicated would improve the reproducibility of the review.
[Figure 8] The Venn diagram labels D, H, and M (Datalog, Horn, and modal logic) are not defined in the caption; please add them so the figure is self-contained.
[Appendix A.2 and A.3] Tables B and C are the empirical backbone of the paper and are already machine-checkable in style; publishing them as CSV/JSON alongside the paper would ease verification and reuse.
[Section 5.1] The claim that 'all logic formalisms currently in use for SKE are essentially particular cases of propositional logic' is asserted without a formal argument; a brief justification or a reference would help readers assess it.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the taxonomies are induced from 249 external primary works; the few self-citations supply background or starting dimensions but are not load-bearing, and the SKI 'expert knowledge' category is an internal consistency issue rather than a circular derivation.

full rationale

Walking the claimed derivation chain, the paper's central deliverables are taxonomies and meta-models, not predictions derived from fitted parameters or equations. Section 3.2 describes a reproducible literature-selection process (keyword queries on five bibliographic engines, first two pages of results, snowballing through 11 secondary works) yielding 249 primary works, and Section 4 states that 'we let taxonomies emerge from the literature rather than super-imposing any particular view of ours.' The classifications in Tables B and C assign external methods to inductively identified dimensions; no quantity is fitted to a subset of data and then reported as a prediction, and no equation equates an output to an input by construction. The only self-citations entering the derivation are [42] in Section 4.1 ('By building upon secondary works, such as the work by [42] and the survey of [6], we identify three relevant dimensions...') and [60,61] in Section 2.3 for the notion of interpretation. These are background or starting points: the dimensions are corroborated by the external survey [6] and are checked against the 132-method SKE corpus, so the taxonomy does not reduce to the authors' prior work. The skeptic's point about 'expert knowledge' identifies a genuine consistency defect: Section 3.1.2 requires SKI input knowledge to be machine-interpretable ('constraining the input knowledge to be machine-interpretable as well'), while Section 4.2.1 defines expert knowledge as 'any piece of human- (but not necessarily machine-) interpretable knowledge' and Table C labels several methods with input 'E'. That contradicts the paper's own definition and weakens the taxonomy's validity, but it is an internal inconsistency rather than a circular reduction, so it does not raise the circularity score. Overall, the derivation is self-contained against the surveyed literature; the score of 2 reflects only minor, non-load-bearing self-citations.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters or invented entities are introduced. The central inputs are the stipulated definitions of SKE and SKI, the representative-sample assumption, and the qualitative coding of surveyed methods.

assumptions (3)

ad hoc to paper The definitions of SKE and SKI in Section 3.1 are accepted as stipulated boundaries for which methods enter the survey.
These are the authors' own broad definitions, not derived from an external standard, and they determine the inclusion of all 249 primary works.
domain assumption The 249 selected primary works are representative of the full SKE and SKI literature.
The search used five keyword queries and the first two pages of results per engine, then snowballed from 11 secondary works. Section 5.5 acknowledges that different terminology may have caused missed works.
domain assumption The qualitative classification of each method into the taxonomy categories is accurate.
Classification was performed by the authors and is reported without inter-rater validation or an independent replication check.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Symbolic Knowledge Extraction and Injection with Sub-symbolic Predictors: A Systematic Literature Review." pith.science (2026). https://pith.science/paper/OJAMLMRF

@misc{pith2026250114836,
  author       = {Pith},
  title        = {Pith review of: Symbolic Knowledge Extraction and Injection with Sub-symbolic Predictors: A Systematic Literature Review},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OJAMLMRF}},
  note         = {Machine review of arXiv:2501.14836}
}

read the original abstract

In this paper we focus on the opacity issue of sub-symbolic machine learning predictors by promoting two complementary activities, namely, symbolic knowledge extraction (SKE) and injection (SKI) from and into sub-symbolic predictors. We consider as symbolic any language being intelligible and interpretable for both humans and computers. Accordingly, we propose general meta-models for both SKE and SKI, along with two taxonomies for the classification of SKE and SKI methods. By adopting an explainable artificial intelligence (XAI) perspective, we highlight how such methods can be exploited to mitigate the aforementioned opacity issue. Our taxonomies are attained by surveying and classifying existing methods from the literature, following a systematic approach, and by generalising the results of previous surveys targeting specific sub-topics of either SKE or SKI alone. More precisely, we analyse 132 methods for SKE and 117 methods for SKI, and we categorise them according to their purpose, operation, expected input/output data and predictor types. For each method, we also indicate the presence/lack of runnable software implementations. Our work may be of interest for data scientists aiming at selecting the most adequate SKE/SKI method for their needs, and also work as suggestions for researchers interested in filling the gaps of the current state of the art, as well as for developers willing to implement SKE/SKI-based technologies.

Figures

Figures reproduced from arXiv: 2501.14836 by the authors.

**Figure 9.** Overview of major strategies followed by surveyed SKI methods. [PITH_FULL_IMAGE:figures/full_fig_p024_9.png] view at source ↗

**Figure 14.** Summary of SKE and SKI taxonomies derived from the literature, as discussed in section 4. [PITH_FULL_IMAGE:figures/full_fig_p027_14.png] view at source ↗

Figures from the paper (1 more)

**Figure 15.** Figure 15: ML workflow enriched with SKI and SKE phases. On the right, the train-extract-fix-inject loop is [PITH_FULL_IMAGE:figures/full_fig_p031_15.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

298 extracted references · 94 canonical work pages

[42]

Roberta Calegari, Giovanni Ciatto, and Andrea Omicini. 2020. On the integration of symbolic and sub-symbolic techniques for XAI: A survey. Intelligenza Artificiale 14, 1 (2020), 7–32. https://doi.org/10.3233/IA-190036

doi:10.3233/ia-190036 2020
[1]

From Objects to Agents

Andrea Agiollo, Giovanni Ciatto, and Andrea Omicini. 2021. Graph Neural Networks as the Copula Mundi between Logic and Machine Learning: A Roadmap. In WOA 2021 – 22nd Workshop “From Objects to Agents” (Bologna, Italy) (CEUR Workshop Proceedings, Vol. 2963) , Roberta Calegari, Giovanni Ciatto, Enrico Denti, Andrea Omicini, and Giovanni Sartor (Eds.). CEUR-...

2021
[2]

Andrea Agiollo and Andrea Omicini. 2022. GNN2GNN: Graph Neural Networks to Generate Neural Networks. In Uncertainty in Artificial Intelligence (Proceedings of Machine Learning Research, Vol. 180) . ML Research Press, 32–42. https://proceedings.mlr.press/v180/agiollo22a.html Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intellige...

2022
[3]

Kareem Ahmed, Kai-Wei Chang, and Guy Van den Broeck. 2023. Semantic Strengthening of Neuro-Symbolic Learning. CoRR abs/2302.14207 (2023). https://doi.org/10.48550/arXiv.2302.14207 arXiv:2302.14207

work page Pith review arXiv doi:10.48550/arxiv.2302.14207 2023
[4]

Mikós Ajtai and Yuri Gurevich. 1994. Datalog vs First-Order Logic. J. Comput. System Sci. 49, 3 (1994), 562–588. https://doi.org/10.1016/S0022-0000(05)80071-6

work page doi:10.1016/s0022-0000(05)80071-6 1994
[5]

Miltiadis Allamanis, Pankajan Chanthirasegaran, Pushmeet Kohli, and Charles Sutton. 2017. Learning Continuous Semantic Representations of Symbolic Expressions. In Proceedings of the 34th International Conference on Machine Learning (ICML), Sydney, NSW, Australia, August 6-11 2017 (Proceedings of Machine Learning Research, Vol. 70) , Doina Precup and Yee W...

2017
[6]

Robert Andrews, Joachim Diederich, and Alan B. Tickle. 1995. Survey and critique of techniques for extracting rules from trained artificial neural networks. Knowledge-Based Systems 8, 6 (1995), 373–389. https://doi.org/10.1016/0950- 7051(96)81920-4

doi:10.1016/0950- 1995
[7]

Robert Andrews and Shlomo Geva. 1995. RULEX & CEBP Networks As the Basis for a Rule Refinement System. In Hybrid Problems, Hybrid Solutions , J. Hallam (Ed.). IOS Press, 1–12

1995

Show all 298 references

[8]

Gethsiyal Augasta and T

M. Gethsiyal Augasta and T. Kathirvalavakumar. 2012. Reverse Engineering the Neural Networks for Rule Extraction in Classification Problems. Neural Process. Lett. 35, 2 (2012), 131–150. https://doi.org/10.1007/s11063-011-9207-8

2012 doi
[9]

Franz Baader. 2003. Basic description logics. In The Description Logic Handbook: Theory, Implementation, and Applica- tions. Cambridge University Press, USA, 43–95

2003
[10]

Sebastian Bader. 2009. Extracting Propositional Rules from Feedforward Neural Networks by Means of Binary Decision Diagrams. In NeSy’09 – Neural-Symbolic Learning and Reasoning (CEUR Workshop Proceedings, Vol. 481) , Artur S. d’Avila Garcez and Pascal Hitzler (Eds.). CEUR-WS.o...

2009
[11]

d’Avila Garcez, and Pascal Hitzler

Sebastian Bader, Artur S. d’Avila Garcez, and Pascal Hitzler. 2005. Computing First-Order Logic Programs by Fibring Artificial Neural Networks. In Proceedings of the 18th International Florida Artificial Intelligence Research Society Conference (FlAIRS), Clearwater Beach, Flor...

2005
[12]

Sebastian Bader, Steffen Hölldobler, and Nuno C. Marques. 2008. Guiding Backprop by Inserting Rules. In NeSy’08 – Neural-Symbolic Learning and Reasoning (CEUR Workshop Proceedings, Vol. 366) , Artur S. d’Avila Garcez and Pascal Hitzler (Eds.). CEUR-WS.org, Patras, Greece. http...

2008
[13]

Sebastian Bader, Steffen Hölldobler, and Valentin Mayer-Eichberger. 2007. Extracting Propositional Rules from Feed-forward Neural Networks – A New Decompositional Approach. In NeSy’07 – Neural-Symbolic Learning and Reasoning, 3rd International Workshop (CEUR Workshop Proceedin...

2007
[14]

Samy Badreddine, Artur d’Avila Garcez, Luciano Serafini, and Michael Spranger. 2020. Logic Tensor Networks. CoRR abs/2012.13635 (2020). arXiv:2012.13635 https://arxiv.org/abs/2012.13635

2020 arXiv
[15]

Luyi Bai, Wenting Yu, Die Chai, Wenjun Zhao, and Mingzhuo Chen. 2023. Temporal knowledge graphs reasoning with iterative guidance by temporal logical rules. Inf. Sci. 621 (2023), 22–35. https://doi.org/10.1016/j.ins.2022.11.096 82https://github.com/nlp-tlp/cyle ACM Comput. Sur...

2023 doi
[16]

Dana H. Ballard. 1986. Parallel Logical Inference and Energy Minimization. In Proceedings of the 5th National Conference on Artificial Intelligence. Philadelphia, PA, USA, August 11-15, 1986. Volume 1: Science , Tom Kehler (Ed.). Morgan Kaufmann, 203–209. http://www.aaai.org/L...

1986
[17]

Barakat and Andrew P

Nahla H. Barakat and Andrew P. Bradley. 2007. Rule Extraction from Support Vector Machines: A Sequential Covering Approach. IEEE Transactions on Knowledge and Data Engineering 19, 6 (2007), 729–741. https://doi.org/10.1109/TKDE. 2007.190610

2007
[18]

Barakat and Joachim Diederich

Nahla H. Barakat and Joachim Diederich. 2005. Eclectic Rule-Extraction from Support Vector Machines. International Journal of Computer and Information Engineering 2, 5 (2005), 1672–1675. https://doi.org/10.5281/zenodo.1055511

2005 doi
[19]

Alberto Barbado, Óscar Corcho, and Richard Benjamins. 2022. Rule extraction in unsupervised anomaly detection for model explainability: Application to OneClass SVM. Expert Syst. Appl. 189 (2022), 116100. https://doi.org/10.1016/j. eswa.2021.116100

2022
[20]

Pietro Barbiero, Gabriele Ciravegna, Francesco Giannini, Pietro Liò, Marco Gori, and Stefano Melacci. 2022. Entropy- Based Logic Explanations of Neural Networks. In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Appli...

2022
[21]

Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garcia, Sergio Gil-Lopez, Daniel Molina, Richard Benjamins, Raja Chatila, and Francisco Herrera. 2020. Explainable Explainable Artificial Intelligence (XA...

2020 doi
[22]

Rodrigo Basilio, Gerson Zaverucha, and Valmir Carneiro Barbosa. 2001. Learning Logic Programs with Neural Networks. In Inductive Logic Programming, 11th International Conference, ILP 2001, Strasbourg, France, September 9-11, 2001, Proceedings (Lecture Notes in Computer Science...

2001 doi
[23]

Anson Bastos, Abhishek Nadgeri, Kuldeep Singh, Isaiah Onando Mulang’, Saeedeh Shekarpour, Johannes Hoffart, and Manohar Kaul. 2021. RECON: Relation Extraction using Knowledge Graph Context in a Graph Neural Network. In WWW ’21: The Web Conference 2021, Virtual Event / Ljubljan...

2021 doi
[24]

José Manuel Benítez, Juan Luis Castro, and Ignacio Requena. 1997. Are artificial neural networks black boxes? IEEE Transactions on Neural Networks 8, 5 (1997), 1156–1164. https://doi.org/10.1109/72.623216

1997 doi
[25]

Hamid R. Berenji. 1991. Refinement of Approximate Reasoning-based Controllers by Reinforcement Learning. In Proceedings of the Eighth International Workshop (ML91), Northwestern University, Evanston, Illinois, USA , Lawrence Birnbaum and Gregg Collins (Eds.). Morgan Kaufmann, ...

1991 doi
[26]

Besold, Artur S

Tarek R. Besold, Artur S. d’Avila Garcez, Sebastian Bader, Howard Bowman, Pedro M. Domingos, Pascal Hitzler, Kai-Uwe Kühnberger, Luís C. Lamb, Daniel Lowd, Priscila Machado Vieira Lima, Leo de Penning, Gadi Pinkas, Hoifung Poon, and Gerson Zaverucha. 2017. Neural-Symbolic Lear...

2017 arXiv
[27]

Jiang Bian, Bin Gao, and Tie-Yan Liu. 2014. Knowledge-Powered Deep Learning for Word Embedding. In Proceedings of the 25th Machine Learning and Knowledge Discovery in Databases - European Conference (ECML), Nancy, France, September 15-19, 2014 (Lecture Notes in Computer Scienc...

2014 doi
[28]

Biswas, Manomita Chakraborty, Biswajit Purkayastha, Pinki Roy, and Dalton Meitei Thounaojam

Saroj K. Biswas, Manomita Chakraborty, Biswajit Purkayastha, Pinki Roy, and Dalton Meitei Thounaojam. 2017. Rule Extraction from Training Data Using Neural Network. Int. J. Artif. Intell. Tools 26, 3 (2017), 1750006:1–1750006:26. https://doi.org/10.1142/S0218213017500063

2017 doi
[29]

Guido Bologna. 2021. A Rule Extraction Technique Applied to Ensembles of Neural Networks, Random Forests, and Gradient-Boosted Trees. Algorithms 14, 12 (2021), 339. https://doi.org/10.3390/a14120339

2021 doi
[30]

Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. 2012. Joint Learning of Words and Meaning Representations for Open-Text Semantic Parsing. In Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2012, La Palma,...

2012
[31]

Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. 2014. A semantic matching energy function for learning with multi-relational data - Application to word-sense disambiguation. Mach. Learn. 94, 2 (2014), 233–259. https://doi.org/10.1007/s10994-013-5363-6

2014 doi
[32]

Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating Embeddings for Modeling Multi-relational Data. In Proceedings of 27th Annual Conference on Neural Information ACM Comput. Surv., Vol. 56, No. 6, Article 161. Publicatio...

2013
[33]

Antoine Bordes, Jason Weston, Ronan Collobert, and Yoshua Bengio. 2011. Learning Structured Embeddings of Knowledge Bases. In Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2011, San Francisco, California, USA, August 7-11, 2011 , Wolfram Burg...

2011
[34]

Antoine Bosselut, Ronan Le Bras, and Yejin Choi. 2021. Dynamic Neuro-Symbolic Knowledge Graph Construction for Zero-shot Commonsense Question Answering. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of...

2021
[35]

Olcay Boz. 2002. Converting A Trained Neural Network To a Decision Tree DecText - Decision Tree Extractor. In Proceedings of the 2002 International Conference on Machine Learning and Applications - ICMLA 2002, June 24-27, 2002, Las Vegas, Nevada, USA, M. Arif Wani, Hamid R. Ar...

2002
[36]

Brachman and Hector J

Ronald J. Brachman and Hector J. Levesque. 2004. The Tradeoff between Expressiveness and Tractability. InKnowledge Representation and Reasoning, Ronald J. Brachman and Hector J. Levesque (Eds.). Morgan Kaufmann, San Francisco, 327–348. https://doi.org/10.1016/B978-155860932-7/50101-1

2004 doi
[37]

Stone, and Richard A

Leo Breiman, Jerome Friedman, Charles J. Stone, and Richard A. Olshen. 1984. Classification and Regression Trees. CRC Press

1984
[38]

Hudson, David C

Antony Browne, Brian D. Hudson, David C. Whitley, Martyn G. Ford, and Philip Picton. 2004. Biological data mining with neural networks: implementation and application of a flexible decision tree extraction algorithm to genomic problem domains. Neurocomputing 57 (March 2004), 2...

2004 doi
[39]

Clifford Brunk and Michael J. Pazzani. 1991. An Investigation of Noise-Tolerant Relational Concept Learning Algorithms. In Proceedings of the Eighth International Workshop (ML91), Northwestern University, Evanston, Illinois, USA, Lawrence Birnbaum and Gregg Collins (Eds.). Mor...

1991 doi
[40]

Jenna Burrell. 2016. How the machine ‘thinks’: Understanding opacity in machine learning algorithms. Big Data & Society 3, 1 (2016), 12 pages. https://doi.org/10.1177/2053951715622512

2016 doi
[41]

Victor R Basili1 Gianluigi Caldiera and H Dieter Rombach. 1994. The goal question metric approach. In Encyclopedia of software engineering, John J. Marciniak (Ed.). John Wiley & Sons, Inc., 528–532

1994
[44]

Castillo, Antonio González Muñoz, and Raúl Pérez

Luis A. Castillo, Antonio González Muñoz, and Raúl Pérez. 2001. Including a simplicity criterion in the selection of the best rule in a genetic fuzzy learning algorithm. Fuzzy Sets and Systems 120, 2 (2001), 309–321. https: //doi.org/10.1016/S0165-0114(99)00095-0

2001 doi
[45]

Biswas, and Biswajit Purkayastha

Manomita Chakraborty, Saroj K. Biswas, and Biswajit Purkayastha. 2018. Recursive Rule Extraction from NN using Reverse Engineering Technique. New Gener. Comput. 36, 2 (2018), 119–142. https://doi.org/10.1007/S00354-018-0031-9

2018 doi
[46]

Manomita Chakraborty, Saroj Kumar Biswas, and Biswajit Purkayastha. 2020. Rule extraction from neural network trained using deep belief network and back propagation. Knowl. Inf. Syst. 62, 9 (2020), 3753–3781. https://doi.org/10. 1007/s10115-020-01473-0

2020
[47]

Veronica Chan and Christine W. Chan. 2017. Towards Developing the Piece-Wise Linear Neural Network Algorithm for Rule Extraction. Int. J. Cogn. Informatics Nat. Intell. 11, 2 (2017), 57–73. https://doi.org/10.4018/IJCINI.2017040104

2017 doi
[48]

Veronica K. H. Chan and Christine W. Chan. 2020. Towards explicit representation of an artificial neural network model: Comparison of two artificial neural network rule extraction approaches. Petroleum 6, 4 (2020), 329–339. https://doi.org/10.1016/j.petlm.2019.11.005 SI: Artif...

2020 doi
[49]

Kai-Wei Chang, Wen-tau Yih, Bishan Yang, and Christopher Meek. 2014. Typed Tensor Decomposition of Knowledge Bases for Relation Extraction. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, October 25-29, 2014 , Alessand...

2014 doi
[50]

Ming-Wei Chang, Lev-Arie Ratinov, and Dan Roth. 2007. Guiding Semi-Supervision with Constraint-Driven Learning. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (ACL), June 23-30, 2007, Prague, Czech Republic, John A. Carroll, Antal va...

2007
[51]

Tenenbaum

Michael Chang, Tomer Ullman, Antonio Torralba, and Joshua B. Tenenbaum. 2017. A Compositional Object-Based Approach to Learning Physical Dynamics. InProceedings of the 5th International Conference on Learning Representations (ICLR), Toulon, France, April 24-26, 2017 . OpenRevi...

2017
[52]

Chaves, Marley M

Adriana da Costa F. Chaves, Marley M. B. R. Vellasco, and Ricardo Tanscheit. 2005. Fuzzy Rule Extraction from Support Vector Machines. In 5th International Conference on Hybrid Intelligent Systems (HIS 2005), 6-9 November 2005, Rio de Janeiro, Brazil , Nadia Nedjah, Luiza de M...

2005 doi
[53]

Kale, Wenzhe Li, Mohammad Taha Bahadori, and Yan Liu

Zhengping Che, David C. Kale, Wenzhe Li, Mohammad Taha Bahadori, and Yan Liu. 2015. Deep Computational Phenotyping. In Proceedings of the 21th International Conference on Knowledge Discovery and Data Mining (KDD), Sydney, NSW, Australia, August 10-13, 2015, Longbing Cao, Cheng...

2015
[54]

Fei Chen. 2004. Learning Accurate And Understandable Rules From SVM Classifiers . Master’s thesis. Simon Fraser University

2004
[55]

Simin Chen, Soroush Bateni, Sampath Grandhi, Xiaodi Li, Cong Liu, and Wei Yang. 2020. DENAS: Automated rule generation by knowledge extraction from neural networks. In ESEC/FSE ’20: 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Sof...

2020
[56]

Zhenyu Chen, Jianping Li, and Liwei Wei. 2007. A multiple kernel support vector machine scheme for feature selection and rule extraction from gene expression data of cancer tissue. Artificial Intelligence in Medicine 41, 2 (2007), 161–175. https://doi.org/10.1016/j.artmed.2007.07.008

2007 doi
[57]

Kewei Cheng, Ziqing Yang, Ming Zhang, and Yizhou Sun. 2021. UniKER: A Unified Framework for Combining Embedding and Definite Horn Rule Reasoning for Knowledge Graph Inference. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021...

2021
[58]

Stewart, and Jimeng Sun

Edward Choi, Mohammad Taha Bahadori, Le Song, Walter F. Stewart, and Jimeng Sun. 2017. GRAM: Graph-based Attention Model for Healthcare Representation Learning. In Proceedings of the 23rd ACM International Conference on Knowledge Discovery and Data Mining (SIGKDD), Halifax, NS...

2017
[59]

Giovanni Ciatto, Roberta Calegari, Andrea Omicini, and Davide Calvaresi. 2019. Towards XMAS: eXplainability through Multi-Agent Systems. In AI&IoT 2019 – Artificial Intelligence and Internet of Things 2019 (CEUR Workshop Proceedings, Vol. 2502), Claudio Savaglio, Giancarlo For...

2019
[60]

Schumacher, and Andrea Omicini

Giovanni Ciatto, Davide Calvaresi, Michael I. Schumacher, and Andrea Omicini. 2020. An Abstract Framework for Agent-Based Explanations in AI. In 19th International Conference on Autonomous Agents and MultiAgent Systems (Auckland, New Zeland), Amal El Fallah Seghrouchni, Gita S...

2020
[61]

Schumacher, Andrea Omicini, and Davide Calvaresi

Giovanni Ciatto, Michael I. Schumacher, Andrea Omicini, and Davide Calvaresi. 2020. Agent-Based Explanations in AI: Towards an Abstract Framework. InExplainable, Transparent Autonomous Agents and Multi-Agent Systems, Davide Calvaresi, Amro Najjar, Michael Winikoff, and Kary Fr...

2020 doi
[62]

Philipp Cimiano. 2006. Ontology Learning and Population from Text . Springer US. https://doi.org/10.1007/978-0-387- 39252-3

2006 doi
[63]

Gabriele Ciravegna, Pietro Barbiero, Francesco Giannini, Marco Gori, Pietro Liò, Marco Maggini, and Stefano Melacci
[64]

Peter Clark and Tim Niblett. 1989. The CN2 Induction Algorithm. Mach. Learn. 3 (1989), 261–283. https://doi.org/10. 1007/BF00116835

1989
[65]

William W. Cohen. 1993. Efficient Pruning Methods for Separate-and-Conquer Rule Learning Systems. In Proceedings of the 13th International Joint Conference on Artificial Intelligence. Chambéry, France, August 28 - September 3, 1993 , Ruzena Bajcsy (Ed.). Morgan Kaufmann, 988–994

1993
[66]

William W. Cohen. 1995. Fast Effective Rule Induction. In Machine Learning, Proceedings of the Twelfth International Conference on Machine Learning, Tahoe City, California, USA, July 9-12, 1995 , Armand Prieditis and Stuart J. Russell ACM Comput. Surv., Vol. 56, No. 6, Article...

1995 doi
[67]

Craven and Jude W

Mark W. Craven and Jude W. Shavlik. 1994. Using Sampling and Queries to Extract Rules from Trained Neural Networks. In Machine Learning Proceedings 1994. Elsevier, 37–45. https://doi.org/10.1016/B978-1-55860-335-6.50013-1

1994 doi
[68]

Craven and Jude W

Mark W. Craven and Jude W. Shavlik. 1996. Extracting Tree-Structured Representations of Trained Networks. In Advances in Neural Information Processing Systems 8. Proceedings of the 1995 Conference , David S. Touretzky, Michael C. Mozer, and Michael E. Hasselmo (Eds.). The MIT ...

1996
[69]

Daniel Cunnington, Mark Law, Jorge Lobo, and Alessandra Russo. 2023. FFNSL: Feed-Forward Neural-Symbolic Learner. Mach. Learn. 112, 2 (2023), 515–569. https://doi.org/10.1007/s10994-022-06278-6

2023 doi
[70]

Claudia d’Amato, Nicola Flavio Quatraro, and Nicola Fanizzi. 2021. Injecting Background Knowledge into Embedding Models for Predictive Tasks on Knowledge Graphs. In The Semantic Web - 18th International Conference, ESWC 2021, Virtual Event, June 6-10, 2021, Proceedings (Lectur...

2021
[71]

Alessandro Daniele and Luciano Serafini. 2019. Knowledge Enhanced Neural Networks. In PRICAI 2019: Trends in Artificial Intelligence - 16th Pacific Rim International Conference on Artificial Intelligence, Cuvu, Yanuca Island, Fiji, August 26-30, 2019, Proceedings, Part I (Lect...

2019 doi
[72]

Tirtharaj Dash, Ashwin Srinivasan, and A. Baskar. 2022. Inclusion of domain-knowledge into GNNs using mode- directed inverse entailment. Mach. Learn. 111, 2 (2022), 575–623. https://doi.org/10.1007/s10994-021-06090-8

2022 doi
[73]

Tirtharaj Dash, Ashwin Srinivasan, and Lovekesh Vig. 2021. Incorporating symbolic domain knowledge into graph neural networks. Machine Learning 110, 7 (2021), 1609–1636. https://doi.org/10.1007/s10994-021-05966-z

2021 doi
[74]

Biswas, Manomita Chakraborty, and Sunita Sarkar

Abhinaba Dattachaudhuri, Saroj K. Biswas, Manomita Chakraborty, and Sunita Sarkar. 2021. A transparent rule-based expert system using neural network. Soft Computing 25, 12 (2021), 7731–7744. https://doi.org/10.1007/s00500-020- 05547-7

2021 doi
[75]

d’Avila Garcez, Krysia Broda, and Dov M

Artur S. d’Avila Garcez, Krysia Broda, and Dov M. Gabbay. 2001. Symbolic knowledge extraction from trained neural networks: A sound approach. Artificial Intelligence 125, 1-2 (2001), 155–207. https://doi.org/10.1016/S0004- 3702(00)00077-1

2001 doi
[76]

d’Avila Garcez and Dov M

Artur S. d’Avila Garcez and Dov M. Gabbay. 2004. Fibring Neural Networks. In Proceedings of the Nineteenth National Conference on Artificial Intelligence, Sixteenth Conference on Innovative Applications of Artificial Intelligence, July 25-29, 2004, San Jose, California, USA , ...

2004
[77]

d’Avila Garcez and Gerson Zaverucha

Artur S. d’Avila Garcez and Gerson Zaverucha. 1999. The Connectionist Inductive Learning and Logic Programming System. Appl. Intell. 11, 1 (1999), 59–77. https://doi.org/10.1023/A:1008328630915

1999 doi
[78]

Paulo Vitor de Campos Souza and Edwin Lughofer. 2022. EFNN-NullUni: An evolving fuzzy neural network based on null-uninorm. Fuzzy Sets and Systems 449 (2022), 1–31. https://doi.org/10.1016/j.fss.2022.01.010

2022 doi
[79]

Thomas Demeester, Tim Rocktäschel, and Sebastian Riedel. 2016. Lifted Rule Injection for Relation Embeddings. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), Austin, Texas, USA, November 1-4, 2016, Jian Su, Xavier Carreras, and Kev...

2016 doi
[80]

Hongyue Diao, Yifan Lu, Ansheng Deng, Li Zou, Xiaofeng Li, and Witold Pedrycz. 2022. Convolutional rule inference network based on belief rule-based system using an evidential reasoning approach. Knowledge-Based Systems 237 (2022), 107713. https://doi.org/10.1016/j.knosys.2021.107713

2022
[81]

Michelangelo Diligenti, Marco Gori, and Claudio Saccà. 2017. Semantic-Based Regularization for Learning and Inference. Artificial Intelligence 244 (2017), 143–165. https://doi.org/10.1016/j.artint.2015.08.011

2017 doi
[82]

Michelangelo Diligenti, Soumali Roychowdhury, and Marco Gori. 2017. Integrating Prior Knowledge into Deep Learning. In Proceedings of the 16th IEEE International Conference on Machine Learning and Applications (ICMLA), Cancun, Mexico, December 18-21, 2017 , Xuewen Chen, Bo Luo...

2017 doi
[83]

Honghua Dong, Jiayuan Mao, Tian Lin, Chong Wang, Lihong Li, and Denny Zhou. 2019. Neural Logic Machines. In7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net. https://openreview.net/forum?id=B1xY-hRctX

2019
[84]

Lu-an Dong, Xin Ye, and Guangfei Yang. 2021. Two-stage rule extraction method based on tree ensemble model for interpretable loan evaluation. Inf. Sci. 573 (2021), 46–64. https://doi.org/10.1016/j.ins.2021.05.063

2021 doi
[85]

Xin Dong, Evgeniy Gabrilovich, Geremy Heitz, Wilko Horn, Ni Lao, Kevin Murphy, Thomas Strohmann, Shaohua Sun, and Wei Zhang. 2014. Knowledge vault: a web-scale approach to probabilistic knowledge fusion. In The 20th ACM SIGKDD International Conference on Knowledge Discovery an...

2014
[86]

Mateo Espinosa Zarlenga, Zohreh Shams, and Mateja Jamnik. 2021. Efficient Decompositional Rule Extraction for Deep Neural Networks. CoRR abs/2111.12628 (2021). arXiv:2111.12628 https://arxiv.org/abs/2111.12628

2021 arXiv
[87]

Etchells and Paulo J

Terence A. Etchells and Paulo J. G. Lisboa. 2006. Orthogonal search-based rule extraction (OSRE) for trained neural networks: a practical and efficient approach. IEEE Transactions on Neural Networks 17, 2 (2006), 374–384. https://doi.org/10.1109/TNN.2005.863472

2006
[88]

Richard Evans and Edward Grefenstette. 2018. Learning Explanatory Rules from Noisy Data. Jounal of Artificial Intelligence Research 61 (2018), 1–64. https://doi.org/10.1613/jair.5714

2018 doi
[89]

Miao Fan, Qiang Zhou, Emily Chang, and Thomas Fang Zheng. 2014. Transition-based Knowledge Graph Embedding with Relational Mapping Properties. In Proceedings of the 28th Pacific Asia Conference on Language, Information and Computation, PACLIC 28, Cape Panwa Hotel, Phuket, Thai...

2014
[90]

Yuan Fang, Kingsley Kuan, Jie Lin, Cheston Tan, and Vijay Chandrasekhar. 2017. Object Detection Meets Knowledge Graphs. In Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI), Melbourne, Australia, August 19-25, 2017, Carles Sierra (Ed.). ...

2017 doi
[91]

Jun Feng, Minlie Huang, Mingdong Wang, Mantong Zhou, Yu Hao, and Xiaoyan Zhu. 2016. Knowledge Graph Embedding by Flexible Translation. InPrinciples of Knowledge Representation and Reasoning: Proceedings of the Fifteenth International Conference, KR 2016, Cape Town, South Afric...

2016
[92]

João Ferreira, Manuel de Sousa Ribeiro, Ricardo Gonçalves, and João Leite. 2022. Looking Inside the Black-Box: Logic-based Explanations for Neural Networks. In Proceedings of the 19th International Conference on Principles of Knowledge Representation and Reasoning, KR 2022, Ha...

2022
[93]

Marc Fischer, Mislav Balunovic, Dana Drachsler-Cohen, Timon Gehr, Ce Zhang, and Martin T. Vechev. 2019. DL2: Training and Querying Neural Networks with Logic. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, Califo...

2019
[94]

Manoel V. M. França, Gerson Zaverucha, and Artur S. d’Avila Garcez. 2014. Fast Relational Learning using Bottom Clause Propositionalization with Artificial Neural Networks. Machine Learning 94, 1 (2014), 81–104. https://doi.org/ 10.1007/s10994-013-5392-1

2014 doi
[95]

Alex A. Freitas. 2014. Comprehensible classification models: a position paper. ACM SIGKDD Explorations Newsletter 15, 1 (June 2014), 1–10. https://doi.org/10.1145/2594473.2594475

2014
[96]

LiMin Fu. 1994. Rule Generation from Neural Networks. IEEE Transactions on Systems, Man, and Cybernetics 24, 8 (1994), 1114–1124. https://doi.org/10.1109/21.299696

1994 doi
[97]

Keerthi, Gih Guang Hung, and Liping Goh

Xiuju Fu, ChongJin Ong, S. Keerthi, Gih Guang Hung, and Liping Goh. 2004. Extracting the knowledge embedded in support vector machines. In 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541) , Vol. 1. 291–296. https://doi.org/10.1109/IJCNN.2004.1379916

2004 arXiv
[98]

Bharat Rao

Glenn Fung, Sathyakama Sandilya, and R. Bharat Rao. 2005. Rule extraction from linear support vector machines. In Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Chicago, Illinois, USA, August 21-24, 2005 , Robert Grossma...

2005
[99]

Johannes Fürnkranz and Gerhard Widmer. 1994. Incremental Reduced Error Pruning. InMachine Learning, Proceedings of the Eleventh International Conference, Rutgers University, New Brunswick, NJ, USA, July 10-13, 1994 , William W. Cohen and Haym Hirsh (Eds.). Morgan Kaufmann, 70–...

1994 doi
[100]

Alberto García-Durán, Antoine Bordes, and Nicolas Usunier. 2014. Effective Blending of Two and Three-way Interactions for Modeling Multi-relational Data. In Proceeding of the 25th Machine Learning and Knowledge Discovery in Databases - European Conference (ECML), Nancy, France...

2014 doi
[101]

Francesco Giannini, Michelangelo Diligenti, Marco Maggini, Marco Gori, and Giuseppe Marra. 2023. T-norms driven loss functions for machine learning. Applied Intelligence (2023), 1–15

2023
[102]

Lee Giles and Christian W

C. Lee Giles and Christian W. Omlin. 1993. Rule refinement with recurrent neural networks. In Proceedings of International Conference on Neural Networks (ICNN’88), San Francisco, CA, USA, March 28 - April 1, 1993 . IEEE, 801–806. ACM Comput. Surv., Vol. 56, No. 6, Article 161....

1993
[103]

Eleonora Giunchiglia and Thomas Lukasiewicz. 2021. Multi-Label Classification Neural Networks with Hard Logical Constraints. Journal of Artificial Intelligence Research 72 (2021), 759–818. https://doi.org/10.1613/jair.1.12850

2021 doi
[104]

Goran Glavas and Ivan Vulic. 2018. Explicit Retrofitting of Distributional Word Vectors. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL), Melbourne, Australia, July 15-20, 2018 , Iryna Gurevych and Yusuke Miyao (Eds.). Associati...

2018 doi
[105]

Goodfellow, Yoshua Bengio, and Aaron C

Ian J. Goodfellow, Yoshua Bengio, and Aaron C. Courville. 2016. Deep Learning. MIT Press, Cambridge, MA, USA. http://www.deeplearningbook.org/

2016
[106]

Right to Explanation

Bryce Goodman and Seth Flaxman. 2017. European Union Regulations on Algorithmic Decision-Making and a “Right to Explanation”. AI Magazine 38, 3 (2017), 50–57. https://doi.org/10.1609/aimag.v38i3.2741

2017 doi
[107]

Goodwin and Dina Demner-Fushman

Travis R. Goodwin and Dina Demner-Fushman. 2019. Bridging the Knowledge Gap: Enhancing Question Answering with World and Domain Knowledge. CoRR abs/1910.07429 (2019). arXiv:1910.07429 http://arxiv.org/abs/1910.07429

2019 arXiv
[108]

Marco Gori (Ed.). 2018. Machine Learning: A Constraint Based Approach . Morgan Kaufmann, Cambridge, MA, USA. 560 pages. https://doi.org/10.1016/C2015-0-00237-4

2018 doi
[109]

Cordell Green and Bertram Raphael

C. Cordell Green and Bertram Raphael. 1968. The use of theorem-proving techniques in question-answering systems. In Proceedings of the 23rd ACM national conference (ACM 1968) , Richard B. Blue Sr. and Arthur M. Rosenberg (Eds.). ACM, New York, NY, USA, 169–181. https://doi.org...

1968
[110]

Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2018. A Survey of Methods for Explaining Black Box Models. Comput. Surveys 51, 5 (2018), 1–42. https://doi.org/10.1145/ 3236009

2018
[111]

Shu Guo, Lin Li, Zhen Hui, Lingshuai Meng, Bingnan Ma, Wei Liu, Lihong Wang, Haibin Zhai, and Hong Zhang
[112]

Shu Guo, Quan Wang, Lihong Wang, Bin Wang, and Li Guo. 2016. Jointly Embedding Knowledge Graphs and Logical Rules. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), Austin, Texas, USA, November 1-4, 2016, Jian Su, Xavier Carreras, an...

2016 doi
[113]

Tameru Hailesilassie. 2016. Rule Extraction Algorithm for Deep Neural Networks: A Review. CoRR abs/1610.05267 (2016), 1–6. arXiv:1610.05267 http://arxiv.org/abs/1610.05267

2016 arXiv
[114]

Halgamuge and Manfred Glesner

Saman K. Halgamuge and Manfred Glesner. 1994. Neural networks in designing fuzzy systems for real world applications. Fuzzy Sets and Systems 65, 1 (1994), 1–12. https://doi.org/10.1016/0165-0114(94)90242-9

1994 doi
[115]

Yoichi Hayashi. 1990. A Neural Expert System with Automated Extraction of Fuzzy If-Then Rules. InAdvances in Neural Information Processing Systems 3, [NIPS Conference, Denver, Colorado, USA, November 26-29, 1990] , Richard Lippmann, John E. Moody, and David S. Touretzky (Eds.)...

1990
[116]

Yoichi Hayashi and Naoki Takano. 2020. One-Dimensional Convolutional Neural Networks with Feature Selection for Highly Concise Rule Extraction from Credit Scoring Datasets with Heterogeneous Attributes. Electronics 9, 8 (2020). https://doi.org/10.3390/electronics9081318

2020 doi
[117]

Harrison, P.C

Jieyue He, Hae-Jin Hu, R. Harrison, P.C. Tai, and Yi Pan. 2006. Rule generation for protein secondary structure prediction with support vector machines and decision tree. IEEE Transactions on NanoBioscience 5, 1 (2006), 46–53. https://doi.org/10.1109/TNB.2005.864021

2006
[119]

Tzung-Pei Hong and Jyh-Bin Chen. 1999. Finding relevant attributes and membership functions. Fuzzy Sets and Systems 103, 3 (1999), 389–404. https://doi.org/10.1016/S0165-0114(97)00187-5

1999 doi
[120]

Tzung-Pei Hong and Jyh-Bin Chen. 2000. Processing individual fuzzy attributes for fuzzy rule induction. Fuzzy Sets and Systems 112, 1 (2000), 127–140. https://doi.org/10.1016/S0165-0114(98)00179-1

2000 doi
[121]

Tzung-Pei Hong and Chai-Ying Lee. 1996. Induction of fuzzy rules and membership functions from training examples. Fuzzy Sets and Systems 84, 1 (1996), 33–47. https://doi.org/10.1016/0165-0114(95)00305-3

1996 doi
[122]

Shin-ichi Horikawa, Takeshi Furuhashi, and Yoshiki Uchikawa. 1992. On fuzzy modeling using fuzzy neural networks with the back-propagation algorithm. IEEE Transactions on Neural Networks 3, 5 (1992), 801–806. https://doi.org/10. ACM Comput. Surv., Vol. 56, No. 6, Article 161. ...

1992
[123]

Alfred Horn. 1951. On sentences which are true of direct unions of algebras. Journal of Symbolic Logic 16, 1 (1951), 14–21. https://doi.org/10.2307/2268661

1951 doi
[124]

Vitor A. C. Horta, Ilaria Tiddi, Suzanne Little, and Alessandra Mileo. 2021. Extracting knowledge from Deep Neural Networks through graph analysis. Future Gener. Comput. Syst. 120 (2021), 109–118. https://doi.org/10.1016/j.future. 2021.02.009

2021 doi
[125]

Hovy, and Eric P

Zhiting Hu, Xuezhe Ma, Zhengzhong Liu, Eduard H. Hovy, and Eric P. Xing. 2016. Harnessing Deep Neural Networks with Logic Rules. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL), Berlin, Germany, August 7-12, 2016. The Associatio...

2016 doi
[126]

Zhiting Hu, Zichao Yang, Ruslan Salakhutdinov, and Eric P. Xing. 2016. Deep Neural Networks with Massive Learned Knowledge. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), Austin, Texas, USA, November 1-4, 2016, Jian Su, Xavier Car...

2016 doi
[127]

Johan Huysmans, Bart Baesens, and Jan Vanthienen. 2006. ITER: An Algorithm for Predictive Regression Rule Extraction. In Data Warehousing and Knowledge Discovery (DaWaK 2006) . Springer, 270–279. https://doi.org/10.1007/ 11823728_26

2006
[128]

Johan Huysmans, Bart Baesens, and Jan Vanthienen. 2006. Using Rule Extraction to Improve the Comprehensibility of Predictive Models. Tech. Report 0612. K.U. Leuven. https://doi.org/10.2139/ssrn.961358

2006 doi
[129]

Johan Huysmans, Karel Dejaeger, Christophe Mues, Jan Vanthienen, and Bart Baesens. 2011. An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models. Decision Support Systems 51, 1 (2011), 141–154. https://doi.org/10.1016/j.dss.2010.12.003

2011 doi
[130]

Muhammad Irfan, Jiangbin Zheng, Muhammad Iqbal, Zafar Masood, and Muhammad Hassan Arif. 2022. Knowledge extraction and retention based continual learning by using convolutional autoencoder-based learning classifier system. Inf. Sci. 591 (2022), 287–305. https://doi.org/10.1016...

2022 doi
[131]

Hisao Ishibuchi, Manabu Nii, and Tadahiko Murata. 1997. Linguistic rule extraction from neural networks and genetic-algorithm-based rule selection. In Proceedings of International Conference on Neural Networks (ICNN’97), Houston, TX, USA, June 9-12, 1997 . IEEE, 2390–2395. htt...

1997
[132]

Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Knowledge Graph Embedding via Dynamic Mapping Matrix. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Proc...

2015 doi
[133]

Guoliang Ji, Kang Liu, Shizhu He, and Jun Zhao. 2016. Knowledge Graph Completion with Adaptive Sparse Transfer Matrix. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12-17, 2016, Phoenix, Arizona, USA, Dale Schuurmans and Michael P. Wellma...

2016
[134]

Shichao Jia, Peiwen Lin, Zeyu Li, Jiawan Zhang, and Shixia Liu. 2020. Visualizing surrogate decision trees of convolutional neural networks. J. Vis. 23, 1 (2020), 141–156. https://doi.org/10.1007/s12650-019-00607-z

2020 doi
[135]

Jingchi Jiang, Huanzheng Wang, Jing Xie, Xitong Guo, Yi Guan, and Qiubin Yu. 2020. Medical knowledge embedding based on recursive neural network for multi-disease diagnosis. Artificial Intelligence in Medicine 103 (2020), 101772. https://doi.org/10.1016/j.artmed.2019.101772

2020
[136]

Ulf Johansson, Cecilia Sönströd, Tuwe Löfström, and Henrik Boström. 2022. Rule extraction with guarantees from regression models. Pattern Recognit. 126 (2022), 108554. https://doi.org/10.1016/j.patcog.2022.108554

2022
[137]

DaeEun Kim and Jaeho Lee. 2000. Handling Continuous-Valued Attributes in Decision Tree with Neural Network Modeling. In Machine Learning: ECML 2000 , Ramon López de Mántaras and Enric Plaza (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 211–219

2000
[138]

Angelika Kimmig, Stephen Bach, Matthias Broecheler, Bert Huang, and Lise Getoor. 2012. A Short Introduction to Probabilistic Soft Logic. InProceedings of the NIPS Workshop on Probabilistic Programming: Foundations and Applications December 07, 2012. Mansinghka, Vikash, 1–4. ht...

2012
[139]

Kipf and Max Welling

Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In Proceedings of the 5th International Conference on Learning Representations (ICLR), Toulon, France, April 24-26, 2017 . OpenReview.net. https://openreview.net/forum?id=SJU4ayYgl

2017
[140]

Rikard König, Ulf Johansson, and Lars Niklasson. 2008. G-REX: A Versatile Framework for Evolutionary Data Mining. In 2008 IEEE International Conference on Data Mining Workshops (ICDM 2008 Workshops) . 971–974. https: //doi.org/10.1109/ICDMW.2008.117

2008 doi
[141]

Kotsiantis

Sotiris B. Kotsiantis. 2007. Supervised Machine Learning: A Review of Classification Techniques.Informatica (Slovenia) 31, 3 (2007), 249–268. http://www.informatica.si/index.php/informatica/article/view/148 ACM Comput. Surv., Vol. 56, No. 6, Article 161. Publication date: Marc...

2007
[142]

Krishnan, G

R. Krishnan, G. Sivakumar, and P. Bhattacharya. 1999. Extracting decision trees from trained neural networks.Pattern Recognit. 32, 12 (1999), 1999–2009. https://doi.org/10.1016/S0031-3203(98)00181-2

1999 doi
[143]

Krishnan, G

R. Krishnan, G. Sivakumar, and P. Bhattacharya. 1999. A search technique for rule extraction from trained neural networks. Pattern Recognit. Lett. 20, 3 (1999), 273–280. https://doi.org/10.1016/S0167-8655(98)00145-7

1999 doi
[144]

Lamb, Artur S

Luís C. Lamb, Artur S. d’Avila Garcez, Marco Gori, Marcelo O. R. Prates, Pedro H. C. Avelar, and Moshe Y. Vardi
[145]

Jens Lehmann, Sebastian Bader, and Pascal Hitzler. 2010. Extracting reduced logic programs from artificial neural networks. Appl. Intell. 32, 3 (2010), 249–266. https://doi.org/10.1007/s10489-008-0142-y

2010 doi
[146]

Levesque and Ronald J

Hector J. Levesque and Ronald J. Brachman. 1987. Expressiveness and tractability in knowledge representation and reasoning. Computational Intelligence 3 (1987), 78–93. https://doi.org/10.1111/j.1467-8640.1987.tb00176.x

1987
[147]

In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020 , Christian Bessiere (Ed.)

Graph Neural Networks Meet Neural-Symbolic Computing: A Survey and Perspective. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020 , Christian Bessiere (Ed.). ijcai.org, Yokohama, Japan, 4877–4884. https://doi.org/10.24963/...

2020 doi
[148]

Weidong Li, Rong Peng, and Zhi Li. 2022. Improving knowledge graph completion via increasing embedding interactions. Appl. Intell. 52, 8 (2022), 9289–9307. https://doi.org/10.1007/s10489-021-02947-6

2022 doi
[149]

Weidong Li, Rong Peng, and Zhi Li. 2023. Knowledge Graph Completion by Jointly Learning Structural Features and Soft Logical Rules. IEEE Transactions on Knowledge and Data Engineering 35, 3 (2023), 2724–2735. https: //doi.org/10.1109/TKDE.2021.3108224

2023
[150]

Hang-Cheng Li, Kai-Qing Zhou, Li-Ping Mo, Azlan Mohd Zain, and Feng Qin. 2020. Weighted Fuzzy Production Rule Extraction Using Modified Harmony Search Algorithm and BP Neural Network Framework. IEEE Access 8 (2020), 186620–186637. https://doi.org/10.1109/ACCESS.2020.3029966

2020
[151]

Xiaodan Liang, Zhiting Hu, Hao Zhang, Liang Lin, and Eric P. Xing. 2018. Symbolic Graph Reasoning Meets Convolutions. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems (NeurIPS), Montréal, Canada, December 3-8, ...

2018
[152]

Qika Lin, Rui Mao, Jun Liu, Fangzhi Xu, and Erik Cambria. 2023. Fusing topology contexts and logical rules in language models for knowledge graph completion. Inf. Fusion 90 (2023), 253–264. https://doi.org/10.1016/j.inffus.2022.09.020

2023 doi
[153]

Wei Li, Luyao Zhu, Rui Mao, and Erik Cambria. 2023. Skier: A symbolic knowledge integrated model for conversational emotion recognition. In Proceedings of the AAAI Conference on Artificial Intelligence

2023
[154]

Zachary C. Lipton. 2018. The Mythos of Model Interpretability. Queue 16, 3 (June 2018), 31–57. https://doi.org/10. 1145/3236386.3241340

2018
[155]

Abbass, and Robert I

Bo Liu, Hussein A. Abbass, and Robert I. McKay. 2002. Density-based heuristic for rule discovery with ant-miner. In The 6th Australia-Japan joint workshop on intelligent and evolutionary system , Vol. 184

2002
[156]

Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning Entity and Relation Embeddings for Knowledge Graph Completion. In Proceedings of the 29th Conference on Artificial Intelligence (AAAI), Austin, Texas, USA, January 25-30, 2015 , Blai Bonet and Sven Ko...

2015
[157]

Yuille, Jonathan Huang, and Kevin Murphy

Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan L. Yuille, Jonathan Huang, and Kevin Murphy. 2018. Progressive Neural Architecture Search. InProceedings of the 15th European Conference on Computer Vision (ECCV 2018) – Part I (Lectu...

2018 doi
[158]

Hanxiao Liu, Yuexin Wu, and Yiming Yang. 2017. Analogical Inference for Multi-relational Embeddings. InProceedings of the 34th International Conference on Machine Learning (ICML), Sydney, NSW, Australia, August 6-11, 2017 (Proceedings of Machine Learning Research, Vol. 70) , D...

2017
[159]

Abbass, and Robert I

Bo Liu, Hussein A. Abbass, and Robert I. McKay. 2004. Classification Rule Discovery with Ant Colony Optimization. IEEE Intell. Informatics Bull. 3, 1 (2004), 31–35. http://www.comp.hkbu.edu.hk/%7Ecib/2004/Feb/2004/Feb/cib_vol3no1_ article4.pdf

2004
[160]

Lloyd (Ed.)

John W. Lloyd (Ed.). 1990. Computational Logic. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642- 76274-1

1990 doi
[161]

Tianle Ma and Aidong Zhang. 2018. Multi-view Factorization AutoEncoder with Network Constraints for Multi-omic Integrative Analysis. In Proceedings of the IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Madrid, Spain, December 3-6, 2018 , Huiru Jane Zhe...

2018
[162]

Quan Liu, Hui Jiang, Zhen-Hua Ling, Si Wei, and Yu Hu. 2016. Probabilistic Reasoning via Deep Learning: Neural Association Models. CoRR abs/1603.07704 (2016). arXiv:1603.07704 http://arxiv.org/abs/1603.07704

2016 arXiv
[163]

Matteo Magnini, Giovanni Ciatto, and Andrea Omicini. 2022. KINS: Knowledge Injection via Network Structuring. In CILC 2022 – Italian Conference on Computational Logic (Bologna, Italy) (CEUR Workshop Proceedings, Vol. 3204), Roberta Calegari, Giovanni Ciatto, and Andrea Omicini...

2022
[164]

From Objects to Agents

Matteo Magnini, Giovanni Ciatto, and Andrea Omicini. 2022. A view to a KILL: Knowledge Injection via Lambda Layer. In WOA 2022 – 23rd Workshop “From Objects to Agents” , Angelo Ferrando and Viviana Mascardi (Eds.). CEUR Workshop Proceedings, Vol. 3261. CEUR-WS.org, Genova, Ita...

2022
[165]

Jocelyn Maclure. 2020. The new AI spring: a deflationary view. AI & Society 35, 3 (2020), 747–750. https://doi.org/10. 1007/s00146-019-00912-z

2020
[166]

J. A. Makowsky. 1987. Why horn formulas matter in computer science: Initial structures and generic examples. J. Comput. System Sci. 34, 2 (April–June 1987), 266–292. https://doi.org/10.1016/0022-0000(87)90027-4

1987 doi
[167]

Robin Manhaeve, Sebastijan Dumancic, Angelika Kimmig, Thomas Demeester, and Luc De Raedt. 2021. Neural probabilistic logic programming in DeepProbLog. Artificial Intelligence 298 (2021), 103504. https://doi.org/10.1016/j. artint.2021.103504

2021
[168]

Ghorbani

Samaneh Mahdavifar and Ali A. Ghorbani. 2020. DeNNeS: deep embedded neural network expert system for detecting cyber attacks. Neural Comput. Appl. 32, 18 (2020), 14753–14780. https://doi.org/10.1007/s00521-020-04830-w

2020 doi
[169]

Kenneth Marino, Xinlei Chen, Devi Parikh, Abhinav Gupta, and Marcus Rohrbach. 2021. KRISP: Integrating Implicit and Symbolic Knowledge for Open-Domain Knowledge-Based VQA. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021 . Com...

2021
[170]

Kenneth Marino, Ruslan Salakhutdinov, and Abhinav Gupta. 2017. The More You Know: Using Knowledge Graphs for Image Classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, July 21-26, 2017 . IEEE Computer Socie...

2017 doi
[171]

Emanuele Marconato, Gianpaolo Bontempo, Elisa Ficarra, Simone Calderara, Andrea Passerini, and Stefano Teso. 2023. Neuro Symbolic Continual Learning: Knowledge, Reasoning Shortcuts and Concept Rehearsal. CoRR abs/2302.01242 (2023). https://doi.org/10.48550/arXiv.2302.01242 arX...

2023 doi arXiv
[172]

Urszula Markowska-Kaczmar and Wojciech Trelak. 2003. Extraction of fuzzy rules from trained neural network using evolutionary algorithm. In ESANN 2003, 11th European Symposium on Artificial Neural Networks, Bruges, Belgium, April 23-25, 2003, Proceedings. 149–154. https://www....

2003
[173]

Giuseppe Marra, Francesco Giannini, Michelangelo Diligenti, and Marco Gori. 2019. LYRICS: a General Interface Layer to Integrate AI and Deep Learning. CoRR abs/1903.07534 (2019). arXiv:1903.07534 http://arxiv.org/abs/1903.07534

2019 arXiv
[174]

Urszula Markowska-Kaczmar and Marcin Chumieja. 2004. Discovering the Mysteries of Neural Networks. Int. J. Hybrid Intell. Syst. 1, 3-4 (2004), 153–163. http://content.iospress.com/articles/international-journal-of-hybrid- intelligent-systems/his016

2004
[175]

David Martens, Bart Baesens, and Tony Van Gestel. 2009. Decompositional Rule Extraction from Support Vector Machines by Active Learning. IEEE Transactions on Knowledge and Data Engineering 21, 2 (2009), 178–191. https: //doi.org/10.1109/TKDE.2008.131

2009 doi
[176]

David Martens, Manu De Backer, Raf Haesen, Jan Vanthienen, Monique Snoeck, and Bart Baesens. 2007. Classification With Ant Colony Optimization. IEEE Transactions on Evolutionary Computation 11, 5 (2007), 651–665. https: //doi.org/10.1109/TEVC.2006.890229

2007
[177]

D. V. Marshakov. 2021. Rule extraction from the Artificial Neural Network. In IOP Conference Series: Materials Science and Engineering, Vol. 1029. IOP Publishing, 012127

2021
[178]

Chirs Matthews and Ilona Jagielska. 1995. Fuzzy rule extraction from a trained multilayer neural network. In Proceedings of ICNN’95 - International Conference on Neural Networks , Vol. 2. 744–748 vol.2. https://doi.org/10.1109/ ICNN.1995.487510

1995
[179]

George F. McNulty. 1977. Fragments of first order logic, I: Universal Horn logic. Journal of Symbolic Logic 42, 2 (1977), 221–237. https://doi.org/10.2307/2272123

1977 doi
[180]

Masuoka, N

R. Masuoka, N. Watanabe, A. Kawamura, Y. Owada, and K. Asakawa. 1990. Neurofuzzy systems – Fuzzy inference using a structured neural network. In Proceedings of International Conference on Fuzzy Logic and Neural Networks, Iizuka Japan, July, 1990. 173–177

1990
[181]

Mike Mintz, Steven Bills, Rion Snow, and Daniel Jurafsky. 2009. Distant Supervision for Relation Extraction without Labeled Data. In Proceedings of the 47th Annual Meeting of the Association for Computational Linguistics (ACL) and the 4th International Joint Conference on Natu...

2009
[182]

Mitchell

Thomas M. Mitchell. 1997. Machine Learning (1 ed.). McGraw-Hill, Inc., New York, NY, USA

1997
[183]

Tim Miller. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence 267 (2019), 1–38. https://doi.org/10.1016/j.artint.2018.07.007

2019 doi
[184]

Nikola Mrksic, Diarmuid Ó Séaghdha, Blaise Thomson, Milica Gasic, Lina Maria Rojas-Barahona, Pei-Hao Su, David Vandyke, Tsung-Hsien Wen, and Steve J. Young. 2016. Counter-fitting Word Vectors to Linguistic Constraints. In Proceedings of the Conference of the North American Cha...

2016
[185]

Murphy and Michael J

Patrick M. Murphy and Michael J. Pazzani. 1991. ID2-of-3: Constructive induction of M-of-N concepts for discrimina- tors in decision trees. In Machine Learning Proceedings 1991 . Elsevier, 183–187

1991
[186]

Sushmita Mitra. 1994. Fuzzy MLP based expert system for medical diagnosis. Fuzzy Sets and Systems 65, 2 (1994), 285–296. https://doi.org/10.1016/0165-0114(94)90025-6 Fuzzy Methods for Computer Vision and Pattern Recognition

1994 doi
[187]

Nauck and Rudolf Kruse

Detlef D. Nauck and Rudolf Kruse. 1997. A neuro-fuzzy method to learn fuzzy classification rules from data. Fuzzy Sets and Systems 89, 3 (1997), 277–288. https://doi.org/10.1016/S0165-0114(97)00009-2

1997 doi
[188]

Nauck and Rudolf Kruse

Detlef D. Nauck and Rudolf Kruse. 1999. Neuro-fuzzy systems for function approximation. Fuzzy Sets and Systems 101, 2 (1999), 261–271. https://doi.org/10.1016/S0165-0114(98)00169-9

1999 doi
[189]

Ali Bou Nassif, Ismail Shahin, Imtinan Basem Attili, Mohammad Azzeh, and Khaled Shaalan. 2019. Speech Recognition Using Deep Neural Networks: A Systematic Review. IEEE Access 7 (2019), 19143–19165. https://doi.org/10.1109/ ACCESS.2019.2896880

2019
[190]

Maximilian Nickel, Lorenzo Rosasco, and Tomaso A. Poggio. 2016. Holographic Embeddings of Knowledge Graphs. In Proceedings of the 30th Conference on Artificial Intelligence (AAAI), Phoenix, Arizona, USA, February 12-17, 2016 , Dale Schuurmans and Michael P. Wellman (Eds.). AAA...

2016
[191]

Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2011. A Three-Way Model for Collective Learning on Multi-Relational Data. In Proceedings of the 28th International Conference on Machine Learning (ICML), Bellevue, Washington, USA, June 28 - July 2, 2011 , Lise Getoor an...

2011
[192]

Chau Nguyen, Tim French, Wei Liu, and Michael Stewart. 2023. CylE: Cylinder Embeddings for Multi-hop Reasoning over Knowledge Graphs. In Proceedings of the 17th Conference of the European Chapter of the Association for Compu- tational Linguistics, EACL 2023, Dubrovnik, Croatia...

2023
[193]

Josué Obregon and Jae-Yoon Jung. 2023. RuleCOSI+: Rule extraction for interpreting classification tree ensembles. Information Fusion 89 (2023), 355–381. https://doi.org/10.1016/j.inffus.2022.08.021

2023 doi
[194]

Josué Obregon, Aekyung Kim, and Jae-Yoon Jung. 2019. RuleCOSI: Combination and simplification of production rules from boosted decision trees for imbalanced classification. Expert Syst. Appl. 126 (2019), 64–82. https://doi.org/ 10.1016/J.ESWA.2019.02.012

2019 doi
[195]

Haydemar Núñez, Cecilio Angulo, and Andreu Català. 2008. Rule Extraction Based on Support and Prototype Vectors. In Rule Extraction from Support Vector Machines , Joachim Diederich (Ed.). Studies in Computational Intelligence, Vol. 80. Springer, 109–134. https://doi.org/10.100...

2008 doi
[196]

d’Avila Garcez

Simon Odense and Artur S. d’Avila Garcez. 2020. Layerwise Knowledge Extraction from Deep Convolutional Networks. CoRR abs/2003.09000 (2020). arXiv:2003.09000 https://arxiv.org/abs/2003.09000

2020 arXiv
[197]

Otter, Julian R

Daniel W. Otter, Julian R. Medina, and Jugal K. Kalita. 2021. A Survey of the Usages of Deep Learning for Natural Language Processing. IEEE Transactions on Neural Networks and Learning Systems 32, 2 (2021), 604–624. https: //doi.org/10.1109/TNNLS.2020.2979670

2021
[198]

Koichi Odajima, Yoichi Hayashi, Gong Tianxia, and Rudy Setiono. 2008. Greedy rule generation from discrete data and its use in neural network rule extraction. Neural Networks 21, 7 (2008), 1020–1028. https://doi.org/10.1016/j. neunet.2008.01.003

2008 doi
[199]

Baolin Peng, Chunyuan Li, Zhu Zhang, Jinchao Li, Chenguang Zhu, and Jianfeng Gao. 2021. SYNERGY: Building Task Bots at Scale Using Symbolic Knowledge and Machine Teaching. CoRR abs/2110.11514 (2021). arXiv:2110.11514 https://arxiv.org/abs/2110.11514

2021 arXiv
[200]

Peters, Mark Neumann, Robert L

Matthew E. Peters, Mark Neumann, Robert L. Logan IV, Roy Schwartz, Vidur Joshi, Sameer Singh, and Noah A. Smith
[201]

Parpinelli, Heitor S

Rafael S. Parpinelli, Heitor S. Lopes, and Alex A. Freitas. 2001. An ant colony based system for data mining: applications to medical data. InProceedings of the genetic and evolutionary computation conference (GECCO-2001). Citeseer, 791–797

2001
[202]

Gadi Pinkus. 1991. Constructing Proofs in Symmetric Networks. In Advances in Neural Information Processing Systems 4, [NIPS Conference, Denver, Colorado, USA, December 2-5, 1991] , John E. Moody, Stephen Jose Hanson, and Richard Lippmann (Eds.). Morgan Kaufmann, 217–224. http:...

1991
[203]

Emanoil Pop, Ross Hayward, and Joachim Diederich. 1994. RULENEG: Extracting rules from a trained ANN by stepwise negation. Technical Report. Neurocomputing Research Centre, Queensland University of Technology

1994
[204]

Mohsen Pourvali, Yao Meng, Chen Sheng, and Yangzhou Du. 2023. TaxoKnow: Taxonomy as Prior Knowledge in the Loss Function of Multi-class Classification. CoRR abs/2305.16341 (2023). https://doi.org/10.48550/arXiv.2305.16341 arXiv:2305.16341

2023 doi arXiv
[205]

Gadi Pinkas, Priscila Lima, and Shimon Cohen. 2013. Representing, binding, retrieving and unifying relational knowledge using pools of neural binders. Biologically Inspired Cognitive Architectures 6 (2013), 87–95. https: //doi.org/10.1016/j.bica.2013.07.005 BICA 2013: Papers f...

2013 doi
[206]

Ross Quinlan

J. Ross Quinlan. 1993. C4.5: Programs for Machine Learning . Morgan Kauffmann, San Mateo, CA, USA. https: //dl.acm.org/doi/10.5555/152181

1993 doi
[207]

Rabuñal, Julian Dorado, Alejandro Pazos, Javier Pereira, and Daniel Rivero

Juan R. Rabuñal, Julian Dorado, Alejandro Pazos, Javier Pereira, and Daniel Rivero. 2004. A New Approach to the Extraction of ANN Rules and to Their Generalization Capacity Through GP. Neural Comput. 16, 7 (2004), 1483–1523. https://doi.org/10.1162/089976604323057461

2004 doi
[208]

Hongyu Ren and Jure Leskovec. 2020. Beta Embeddings for Multi-Hop Logical Reasoning in Knowledge Graphs. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, Hugo L...

2020
[209]

Ross Quinlan

J. Ross Quinlan. 1986. Induction of Decision Trees. Mach. Learn. 1, 1 (1986), 81–106. https://doi.org/10.1023/A: 1022643204877

1986 doi
[210]

Tim Rocktäschel and Sebastian Riedel. 2017. End-to-end Differentiable Proving. InAdvances in Neural Information Pro- cessing Systems: Annual Conference on Neural Information Processing Systems, Long Beach, CA, USA, December 4-9, 2017 , Isabelle Guyon, Ulrike von Luxburg, Samy ...

2017
[211]

Tim Rocktäschel, Sameer Singh, and Sebastian Riedel. 2015. Injecting Logical Background Knowledge into Embeddings for Relation Extraction. In The 2015 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL): Human Language Technologies...

2015 doi
[212]

Natalia Díaz Rodríguez, Alberto Lamas, Jules Sanchez, Gianni Franchi, Ivan Donadello, Siham Tabik, David Filliat, Policarpo Cruz, Rosana Montes, and Francisco Herrera. 2022. EXplainable Neural-Symbolic Learning ( X-NeSyL) methodology to fuse deep learning representations with ...

2022 doi
[213]

Gray, Francois P

Ryan Riegel, Alexander G. Gray, Francois P. S. Luus, Naweed Khan, Ndivhuwo Makondo, Ismail Yunus Akhalwaya, Haifeng Qian, Ronald Fagin, Francisco Barahona, Udit Sharma, Shajith Ikbal, Hima Karanam, Sumit Neelam, Ankita Likhyani, and Santosh K. Srivastava. 2020. Logical Neural ...

2020 arXiv
[214]

Cynthia Rudin, Chaofan Chen, Zhi Chen, Haiyang Huang, Lesia Semenova, and Chudi Zhong. 2022. Interpretable machine learning: Fundamental principles and 10 grand challenges. Statistics Surveys 16 (2022), 1–85. https: //doi.org/10.1214/21-SS133

2022 doi
[215]

Rumelhart, Geoffrey E

D. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams. 1986. Learning representations by back-propagating errors. Nature 323 (1986), 533–536. https://doi.org/10.1038/323533a0

1986 doi
[216]

Saad and Donald C

Emad W. Saad and Donald C. Wunsch II. 2007. Neural network explanation using inversion. Neural Networks 20, 1 (2007), 78–93. https://doi.org/10.1016/j.neunet.2006.07.005

2007 doi
[217]

Cynthia Rudin. 2019. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence 1, 5 (2019), 206–215. https://doi.org/10.1038/s42256-019-0048-x

2019 doi
[218]

Federico Sabbatini and Roberta Calegari. 2023. Unveiling Opaque Predictors via Explainable Clustering: The CReEPy Algorithm. In Proceedings of the 2nd BEW ARE workshop, BEW ARE 2023, Rome, Italy. November 6, 2023 (accepted). ACM Comput. Surv., Vol. 56, No. 6, Article 161. Publ...

2023
[219]

Federico Sabbatini, Giovanni Ciatto, and Andrea Omicini. 2021. GridEx: An Algorithm for Knowledge Extraction from Black-Box Regressors. In Explainable and Transparent AI and Multi-Agent Systems. Third International Workshop, EXTRAAMAS 2021, Virtual Event, May 3–7, 2021, Revise...

2021
[220]

Kazumi Saito and Ryohei Nakano. 1988. Medical diagnostic expert system based on PDP model. In Proceedings of International Conference on Neural Networks (ICNN’88), San Diego, CA, USA, July 24-27, 1988 . IEEE, 255–262. https://doi.org/10.1109/ICNN.1988.23855

1988
[221]

Federico Sabbatini and Roberta Calegari. 2022. Symbolic Knowledge Extraction from Opaque Machine Learning Predictors: GridREx & PEDRO. In Proceedings of the 19th International Conference on Principles of Knowledge Represen- tation and Reasoning, KR 2022, Haifa, Israel. July 31...

2022 doi
[222]

Kazumi Saito and Ryohei Nakano. 2002. Extracting regression rules from neural networks. Neural Networks 15, 10 (2002), 1279–1288. https://doi.org/10.1016/S0893-6080(02)00089-8

2002 doi
[223]

Armin Salimi-Badr and Mohammad Mehdi Ebadzadeh. 2022. A novel learning algorithm based on computing the rules’ desired outputs of a TSK fuzzy neural network with non-separable fuzzy rules. Neurocomputing 470 (2022), 139–153. https://doi.org/10.1016/j.neucom.2021.10.103

2022 doi
[224]

Makoto Sato and Hiroshi Tsukimoto. 2001. Rule extraction from neural networks via decision tree induction. In IJCNN’01. International Joint Conference on Neural Networks. Proceedings (Cat. No. 01CH37222) , Vol. 3. IEEE, 1870–1875. https://doi.org/10.1109/IJCNN.2001.938448

2001
[225]

Kazumi Saito and Ryohei Nakano. 1997. Law Discovery using Neural Networks. In Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence, IJCAI 97, Nagoya, Japan, August 23-29, 1997, 2 Volumes . Morgan Kaufmann, 1078–1083. http://ijcai.org/Proceedin...

1997
[226]

Gregor P. J. Schmitz, Chris Aldrich, and François S. Gouws. 1999. ANN-DT: an algorithm for extraction of decision trees from artificial neural networks. IEEE Transactions on Neural Networks 10, 6 (1999), 1392–1401. https://doi.org/ 10.1109/72.809084

1999 doi
[227]

Prithviraj Sen, Breno W. S. R. de Carvalho, Ryan Riegel, and Alexander G. Gray. 2022. Neuro-Symbolic Inductive Logic Programming with Logical Neural Networks. In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Applicat...

2022
[228]

Sabrina Sestito and Tharam S. Dillon. 1994. Automated knowledge acquisition. Prentice Hall

1994
[229]

Fieldsend, Derek Partridge, Timothy J

Vitaly Schetinin, Jonathan E. Fieldsend, Derek Partridge, Timothy J. Coats, Wojtek J. Krzanowski, Richard M. Everson, Trevor C. Bailey, and Adolfo Hernandez. 2007. Confident Interpretation of Bayesian Decision Tree Ensembles for Clinical Applications. IEEE Transactions on Info...

2007
[230]

Rudy Setiono. 1997. Extracting Rules from Neural Networks by Pruning and Hidden-Unit Splitting. Neural Comput. 9, 1 (1997), 205–225. https://doi.org/10.1162/neco.1997.9.1.205

1997 doi
[231]

Rudy Setiono. 2000. Extracting M-of-N rules from trained neural networks. IEEE Transactions on Neural Networks 11, 2 (2000), 512–519. https://doi.org/10.1109/72.839020

2000 doi
[232]

Rudy Setiono, Bart Baesens, and Christophe Mues. 2008. Recursive Neural Network Rule Extraction for Data With Mixed Attributes. IEEE Transactions on Neural Networks19, 2 (2008), 299–307. https://doi.org/10.1109/TNN.2007.908641

2008
[233]

Kamal Kumar Sethi, Durgesh Kumar Mishra, and Bharat Mishra. 2012. KDRuleEx: A Novel Approach for Enhancing User Comprehensibility Using Rule Extraction. In 2012 Third International Conference on Intelligent Systems Modelling and Simulation. 55–60. https://doi.org/10.1109/ISMS.2012.116

2012 doi
[234]

Rudy Setiono, Wee Kheng Leow, and Jacek M. Zurada. 2002. Extraction of rules from artificial neural networks for nonlinear regression. IEEE Transactions on Neural Networks 13, 3 (2002), 564–577. https://doi.org/10.1109/TNN.2002. 1000125

2002 doi
[235]

Rudy Setiono and Huan Liu. 1996. Symbolic Representation of Neural Networks. Computer 29, 3 (1996), 71–77. https://doi.org/10.1109/2.485895

1996 doi
[236]

Rudy Setiono and Huan Liu. 1997. NeuroLinear: A System for Extracting Oblique Decision Rules from Neural Networks. In Machine Learning: ECML-97, 9th European Conference on Machine Learning, Prague, Czech Republic, April 23-25, 1997, Proceedings (Lecture Notes in Computer Scien...

1997 doi
[237]

Rudy Setiono and Wee Kheng Leow. 2000. FERNN: An Algorithm for Fast Extraction of Rules from Neural Networks. Appl. Intell. 12, 1-2 (2000), 15–25. https://doi.org/10.1023/A:1008307919726

2000 doi
[238]

Zohreh Shams, Botty Dimanov, Sumaiyah Kola, Nikola Simidjievski, Helena Andres Terre, Paul Scherer, Urška Matjašec, Jean Abraham, Pietro Liò, and Mateja Jamnik. 2021. REM: An integrative rule extraction methodology for explainable data analysis in healthcare. medRxiv (2021), 2...

2021
[239]

Alisa Smirnova, Jie Yang, Dingqi Yang, and Philippe Cudre-Mauroux. 2022. Nessy: A Neuro-Symbolic System for Label Noise Reduction. IEEE Transactions on Knowledge and Data Engineering (2022)

2022
[240]

Manning, and Andrew Y

Richard Socher, Danqi Chen, Christopher D. Manning, and Andrew Y. Ng. 2013. Reasoning With Neural Tensor Networks for Knowledge Base Completion. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proce...

2013
[241]

Rudy Setiono and James Y. L. Thong. 2004. An approach to generate rules from neural networks for regression problems. Eur. J. Oper. Res. 155, 1 (2004), 239–250. https://doi.org/10.1016/S0377-2217(02)00792-0

2004 doi
[242]

Giuseppe Spillo, Cataldo Musto, Marco de Gemmis, Pasquale Lops, and Giovanni Semeraro. 2022. Knowledge-aware Recommendations Based on Neuro-Symbolic Graph Embeddings and First-Order Logical Rules. InRecSys ’22: Sixteenth ACM Conference on Recommender Systems, Seattle, W A, USA...

2022
[243]

Russell Stewart and Stefano Ermon. 2017. Label-Free Supervision of Neural Networks with Physics and Domain Knowledge. In Proceedings of the 31st Conference on Artificial Intelligence (AAAI), San Francisco, California, USA, February 4-9, 2017, Satinder P. Singh and Shaul Markov...

2017
[244]

Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. 2019. RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space. In Proceedings of the 7th International Conference on Learning Representations (ICLR), New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net...

2019
[245]

Gustav Sourek, Vojtech Aschenbrenner, Filip Zelezný, Steven Schockaert, and Ondrej Kuzelka. 2018. Lifted Relational Neural Networks: Efficient Learning of Latent Relational Structures. Journal of Artificial Intelligence Research 62 (2018), 69–100. https://doi.org/10.1613/jair.1.11203

2018 doi
[246]

Ah-Hwee Tan. 1997. Cascade ARTMAP: Integrating Neural Computation and Symbolic Knowledge Processing. IEEE Transaction on Neural Networks 8, 2 (1997), 237–250. https://doi.org/10.1109/72.557661

1997 doi
[247]

Xiaojuan Tang, Song-Chun Zhu, Yitao Liang, and Muhan Zhang. 2022. RulE: Neural-Symbolic Knowledge Graph Rea- soning with Rule Embedding. CoRR abs/2210.14905 (2022). https://doi.org/10.48550/arXiv.2210.14905 arXiv:2210.14905

2022 doi arXiv
[248]

Sebastian B. Thrun. 1993. Extracting Provably Correct Rules from Artificial Neural Networks . Technical Report. University of Bonn

1993
[249]

Taha and Joydeep Ghosh

Ismail A. Taha and Joydeep Ghosh. 1999. Symbolic Interpretation of Artificial Neural Networks. IEEE Transactions on Knowledge and Data Engineering 11, 3 (1999), 448–463. https://doi.org/10.1109/69.774103

1999 doi
[250]

Gabriele Tolomei, Fabrizio Silvestri, Andrew Haines, and Mounia Lalmas. 2017. Interpretable predictions of tree-based ensembles via actionable feature tweaking. In 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’17) . ACM Press, Halifax, NS...

2017
[251]

Douglas E. D. Torres and Claudio M. S. Rocco. 2005. Extracting Trees from Trained SVM Models using a TREPAN Based Approach. In 5th International Conference on Hybrid Intelligent Systems (HIS 2005), 6-9 November 2005, Rio de Janeiro, Brazil, Nadia Nedjah, Luiza de Macedo Mourel...

2005 doi
[252]

Towell and Jude W

Geoffrey G. Towell and Jude W. Shavlik. 1991. Interpretation of Artificial Neural Networks: Mapping Knowledge-Based Neural Networks into Rules. InAdvances in Neural Information Processing Systems 4, [NIPS Conference, Denver, Colorado, USA, December 2-5, 1991] , John E. Moody, ...

1991
[253]

Tickle, Marian Orlowski, and Joachim Diederich

Alan B. Tickle, Marian Orlowski, and Joachim Diederich. 1996. DEDEC: A methodology for extracting rules from trained artificial neural networks. InRules and Networks: Proceedings of the Rule Extraction from Trained Artificial Neural Networks Workshop, Robert Andrews and Joachi...

1996
[254]

Towell, Jude W

Geoffrey G. Towell, Jude W. Shavlik, and Michiel O. Noordewier. 1990. Refinement ofApproximate Domain Theories by Knowledge-Based Neural Networks. In Proceedings of the 8th National Conference on Artificial Intelligence. Boston, Massachusetts, USA, July 29 - August 3, 1990, 2 ...

1990
[255]

Tran and Artur S

Son N. Tran and Artur S. d’Avila Garcez. 2016. Deep Logic Networks: Inserting and Extracting Knowledge From Deep Belief Networks. IEEE Transaction on Neural Networks and Learning Systems 29, 2 (2016), 246–258. https: //doi.org/10.1109/TNNLS.2016.2603784

2016
[256]

Volker Tresp, Jürgen Hollatz, and Subutai Ahmad. 1992. Network Structuring and Training Using Rule-Based Knowledge. In Advances in Neural Information Processing Systems 5, [NIPS Conference, Denver, Colorado, USA, November ACM Comput. Surv., Vol. 56, No. 6, Article 161. Publica...

1992
[257]

Towell and Jude W

Geoffrey G. Towell and Jude W. Shavlik. 1993. Extracting refined rules from knowledge-based neural networks. Machine Learning 13, 1 (1993), 71–101. https://doi.org/10.1007/BF00993103

1993 doi
[258]

Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. 2016. Complex Embeddings for Simple Link Prediction. In Proceedings of the 33rd International Conference on Machine Learning (ICML), New York City, NY, USA, June 19-24, 2016 (JMLR Workshop...

2016
[259]

Hiroshi Tsukimoto. 2000. Extracting rules from trained neural networks. IEEE Transactions on Neural Networks 11, 2 (2000), 377–389. https://doi.org/10.1109/72.839008

2000 doi
[260]

Bhekisipho Twala. 2010. Multiple classifier application to credit risk assessment. Expert Systems with Applications 37, 4 (2010), 3326–3336

2010
[261]

Volker Tresp, Jürgen Hollatz, and Subutai Ahmad. 1992. Network Structuring and Training Using Rule-Based Knowledge. In Advances in Neural Information Processing Systems 5, [NIPS Conference, Denver, Colorado, USA, November 30 - December 3, 1992] , Stephen Jose Hanson, Jack D. C...

1992
[262]

Tim van Gelder. 1990. Why Distributed Representation is Inherently Non-Symbolic. In Konnektionismus in Artificial Intelligence und Kognitionsforschung (Informatik-Fachberichte, Vol. 252) , Georg Dorffner (Ed.). Springer, Salzburg, Österreich, Austria, 58–66. https://doi.org/10...

1990 doi
[263]

Van Veen and S

F. Van Veen and S. Leijnen. 2019. The Neural Network Zoo. https://www.asimovinstitute.org/neural-network-zoo. [Online; accessed 17-September-2021]

2019
[264]

Nikola Vasilev, Zheni Mincheva, and Ventsislav Nikolov. 2020. Decision Tree Extraction using Trained Neural Network. In Proceedings of the 9th International Conference on Smart Cities and Green ICT Systems, SMARTGREENS 2020, Prague, Czech Republic, May 2-4, 2020 , Cornel Klein...

2020 doi
[265]

Johan Van Benthem and Kees Doets. 2001. Higher-Order Logic. In Handbook of Philosophical Logic , D. M. Gabbay and F. Guenthner (Eds.). Springer Netherlands, Dordrecht, 189–243. https://doi.org/10.1007/978-94-015-9833-0_3

2001 doi
[266]

Paul Voigt and Axel von dem Bussche. 2017. The EU General Data Protection Regulation (GDPR). A Practical Guide . Springer. https://doi.org/10.1007/978-3-319-57959-7

2017 doi
[267]

Laura von Rueden, Sebastian Mayer, Katharina Beckh, Bogdan Georgiev, Sven Giesselbach, Raoul Heese, Birgit Kirsch, Michal Walczak, Julius Pfrommer, Annika Pick, Rajkumar Ramamurthy, Jochen Garcke, Christian Bauckhage, and Jannis Schuecker. 2021. Informed Machine Learning – A T...

2021
[268]

Quan Wang, Zhendong Mao, Bin Wang, and Li Guo. 2017. Knowledge Graph Embedding: A Survey of Approaches and Applications. IEEE Transactions on Knowledge and Data Engineering 29, 12 (2017), 2724–2743. https://doi.org/10. 1109/TKDE.2017.2754499

2017
[269]

Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In Proceedings of the 6th International Conference on Learning Representations (ICLR), Vancouver, BC, Canada, April 30 - May 3, 2018 . OpenRevie...

2018
[270]

Sutong Wang, Yuyan Wang, Dujuan Wang, Yunqiang Yin, Yanzhang Wang, and Yaochu Jin. 2020. An improved random forest-based rule extraction method for breast cancer diagnosis. Applied Soft Computing 86 (2020). https: //doi.org/10.1016/j.asoc.2019.105941

2020
[271]

Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge Graph Embedding by Translating on Hyperplanes. In Proceedings of the 28th Conference on Artificial Intelligence (AAAI), Québec City, Québec, Canada, July 27 -31, 2014, Carla E. Brodley and Peter Stone (Eds...

2014
[272]

Zhepei Wei, Yue Wang, Jinnan Li, Zhining Liu, Erxin Yu, Yuan Tian, Xin Wang, and Yi Chang. 2022. Towards Unified Representations of Knowledge Graph and Expert Rules for Machine Learning and Reasoning. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Asso...

2022
[273]

Quan Wang, Bin Wang, and Li Guo. 2015. Knowledge Base Completion Using Embeddings and Rules. In Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI), Buenos Aires, Argentina, July 25-31, 2015 , Qiang Yang and Michael J. Wooldridge (Eds.). A...

2015
[274]

Luisa Werner, Nabil Layaïda, Pierre Genevès, and Sarah Chlyah. 2023. Knowledge Enhanced Graph Neural Networks. CoRR abs/2303.15487 (2023). https://doi.org/10.48550/arXiv.2303.15487 arXiv:2303.15487

2023 doi arXiv
[275]

Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, and Yejin Choi

Peter West, Chandra Bhagavatula, Jack Hessel, Jena D. Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, and Yejin Choi. 2022. Symbolic Knowledge Distillation: from General Language Models to Commonsense Models. In Proceedings of the 2022 Conference of the North Ameri...

2022
[276]

Wolpert and William G

David H. Wolpert and William G. Macready. 1997. No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation 1, 1 (1997), 67–82. https://doi.org/10.1109/4235.585893

1997
[277]

Zhuoyu Wei, Jun Zhao, Kang Liu, Zhenyu Qi, Zhengya Sun, and Guanhua Tian. 2015. Large-scale Knowledge Base Completion: Inferring via Grounding Network Sampling over Selected Instances. In Proceedings of the 24th International Conference on Information and Knowledge Management ...

2015
[278]

Han Xiao, Minlie Huang, and Xiaoyan Zhu. 2016. From One Point to a Manifold: Knowledge Graph Embedding for Precise Link Prediction. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016, New York, NY, USA, 9-15 July 2016 , Sub...

2016
[279]

Han Xiao, Minlie Huang, and Xiaoyan Zhu. 2016. TransG : A Generative Model for Knowledge Graph Embedding. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers . The Assoc...

2016 doi
[280]

Meel, Mohan S

Yaqi Xie, Ziwei Xu, Kuldeep S. Meel, Mohan S. Kankanhalli, and Harold Soh. 2019. Embedding Symbolic Knowl- edge into Deep Networks. In Advances in Neural Information Processing Systems 32 (NeurIPS 2019) , Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-B...

2019
[281]

Han Xiao, Minlie Huang, Yu Hao, and Xiaoyan Zhu. 2015. TransA: An Adaptive Approach for Knowledge Graph Embedding. CoRR abs/1509.05490 (2015). arXiv:1509.05490 http://arxiv.org/abs/1509.05490

2015 arXiv
[282]

Jingyi Xu, Zilu Zhang, Tal Friedman, Yitao Liang, and Guy Van den Broeck. 2018. A Semantic Loss Function for Deep Learning with Symbolic Knowledge. In Proceedings of the 35th International Conference on Machine Learning (ICML), Stockholmsmässan, Stockholm, Sweden, July 10-15, ...

2018
[283]

Anli Yan, Zhenxiang Chen, Haibo Zhang, Lizhi Peng, Qiben Yan, Muhammad Umair Hassan, Chuan Zhao, and Bo Yang
[284]

Bishan Yang, Wen-Tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding Entities and Relations for Learning and Inference in Knowledge Bases. In Proceedings of the 3rd International Conference on Learning Representations (ICLR), San Diego, CA, USA, May 7-9, 2015 , Yo...

2015 arXiv
[285]

Yaqi Xie, Fan Zhou, and Harold Soh. 2021. Embedding Symbolic Temporal Knowledge into Deep Sequential Models. In IEEE International Conference on Robotics and Automation, ICRA 2021, Xi’an, China, May 30 - June 5, 2021 . IEEE, 4267–4273. https://doi.org/10.1109/ICRA48506.2021.9561952

2021
[286]

Webb, and Xindong Wu

Ying Yang, Geoffrey I. Webb, and Xindong Wu. 2010. Discretization Methods. InData Mining and Knowledge Discovery Handbook, 2nd ed, Oded Maimon and Lior Rokach (Eds.). Springer. https://doi.org/10.1007/978-0-387-09823-4_6

2010 doi
[287]

Dounia Yedjour. 2021. Application of the Genetic Algorithm to the Rule Extraction Problem. In Artificial Intelligence and Renewables Towards an Energy Transition, Mustapha Hatti (Ed.). Springer International Publishing, Cham, 604–611

2021
[288]

Dounia Yedjour and Abdelkader Benyettou. 2018. Symbolic interpretation of artificial neural networks based on multiobjective genetic algorithms and association rules mining. Applied Soft Computing 72 (2018), 177–188. https://doi.org/10.1016/j.asoc.2018.08.007

2018 doi
[289]

Dongran Yu, Bo Yang, Qianhao Wei, Anchen Li, and Shirui Pan. 2022. A Probabilistic Graphical Model Based on Neural-symbolic Reasoning for Visual Relationship Detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-...

2022
[290]

Cheng Yang, Jiawei Liu, and Chuan Shi. 2021. Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Framework. In WWW ’21: The Web Conference 2021, Virtual Event / Ljubljana, Slovenia, April 19-23, 2021, Jure Leskovec, Marko Grobel...

2021
[291]

Yufei Yuan and Huijun Zhuang. 1996. A genetic algorithm for generating fuzzy classification rules. Fuzzy Sets and Systems 84, 1 (1996), 1–19. https://doi.org/10.1016/0165-0114(95)00302-9

1996 doi
[292]

Ying Zhang, Hongye Su, Tao Jia, and Jian Chu. 2005. Rule Extraction from Trained Support Vector Machines. In Advances in Knowledge Discovery and Data Mining, 9th Pacific-Asia Conference, PAKDD 2005, Hanoi, Vietnam, May 18-20, 2005, Proceedings (Lecture Notes in Computer Scienc...

2005 doi
[293]

Zhanqiu Zhang, Jianyu Cai, Yongdong Zhang, and Jie Wang. 2020. Learning Hierarchy-Aware Knowledge Graph Embeddings for Link Prediction. In Proceedings of the 34th Conference on Artificial Intelligence (AAAI), The 32nd Innovative Applications of Artificial Intelligence Conferen...

2020
[294]

Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu. 2019. ERNIE: Enhanced Language Representation with Informative Entities. In Proceedings of the 57th Conference of the Association for Computational Linguistics (ACL), Florence, Italy, July 28 - August 2,...

2019 doi
[295]

Jianbo Yu and Guoliang Liu. 2021. Extracting and Inserting Knowledge into Stacked Denoising Auto-Encoders.Neural Networks 137 (2021), 31–42. https://doi.org/10.1016/j.neunet.2021.01.010 ACM Comput. Surv., Vol. 56, No. 6, Article 161. Publication date: March 2024. Symbolic Know...

2021 doi
[300]

Zhong-Qiu Zhao, Peng Zheng, Shou-tao Xu, and Xindong Wu. 2019. Object Detection With Deep Learning: A Review. IEEE Transactions on Neural Networks and Learning Systems 30, 11 (2019), 3212–3232. https://doi.org/10.1109/TNNLS. 2018.2876865

2019
[2019]

Knowledge Enhanced Contextual Word Representations. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, November 3-7, 2019 , Kentar...

2019 doi
[2020]

Knowledge Graph Embedding Preserving Soft Logical Regularity. In CIKM ’20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland, October 19-23, 2020 , Mathieu d’Aquin, Stefan Dietze, Claudia Hauff, Edward Curry, and Philippe Cud...

2020
[2021]

Neurocomputing 453 (2021), 482–492

Effective detection of mobile malware behavior based on explainable deep neural network. Neurocomputing 453 (2021), 482–492. https://doi.org/10.1016/j.neucom.2020.09.082

2021 doi
[2023]

Logic Explained Networks. Artif. Intell. 314 (2023), 103822. https://doi.org/10.1016/j.artint.2022.103822

2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [42]

Roberta Calegari, Giovanni Ciatto, and Andrea Omicini. 2020. On the integration of symbolic and sub-symbolic techniques for XAI: A survey. Intelligenza Artificiale 14, 1 (2020), 7–32. https://doi.org/10.3233/IA-190036

doi:10.3233/ia-190036 2020

[2] [1]

From Objects to Agents

Andrea Agiollo, Giovanni Ciatto, and Andrea Omicini. 2021. Graph Neural Networks as the Copula Mundi between Logic and Machine Learning: A Roadmap. In WOA 2021 – 22nd Workshop “From Objects to Agents” (Bologna, Italy) (CEUR Workshop Proceedings, Vol. 2963) , Roberta Calegari, Giovanni Ciatto, Enrico Denti, Andrea Omicini, and Giovanni Sartor (Eds.). CEUR-...

2021

[3] [2]

Andrea Agiollo and Andrea Omicini. 2022. GNN2GNN: Graph Neural Networks to Generate Neural Networks. In Uncertainty in Artificial Intelligence (Proceedings of Machine Learning Research, Vol. 180) . ML Research Press, 32–42. https://proceedings.mlr.press/v180/agiollo22a.html Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intellige...

2022

[4] [3]

Kareem Ahmed, Kai-Wei Chang, and Guy Van den Broeck. 2023. Semantic Strengthening of Neuro-Symbolic Learning. CoRR abs/2302.14207 (2023). https://doi.org/10.48550/arXiv.2302.14207 arXiv:2302.14207

work page Pith review arXiv doi:10.48550/arxiv.2302.14207 2023

[5] [4]

Mikós Ajtai and Yuri Gurevich. 1994. Datalog vs First-Order Logic. J. Comput. System Sci. 49, 3 (1994), 562–588. https://doi.org/10.1016/S0022-0000(05)80071-6

work page doi:10.1016/s0022-0000(05)80071-6 1994

[6] [5]

Miltiadis Allamanis, Pankajan Chanthirasegaran, Pushmeet Kohli, and Charles Sutton. 2017. Learning Continuous Semantic Representations of Symbolic Expressions. In Proceedings of the 34th International Conference on Machine Learning (ICML), Sydney, NSW, Australia, August 6-11 2017 (Proceedings of Machine Learning Research, Vol. 70) , Doina Precup and Yee W...

2017

[7] [6]

Robert Andrews, Joachim Diederich, and Alan B. Tickle. 1995. Survey and critique of techniques for extracting rules from trained artificial neural networks. Knowledge-Based Systems 8, 6 (1995), 373–389. https://doi.org/10.1016/0950- 7051(96)81920-4

doi:10.1016/0950- 1995

[8] [7]

Robert Andrews and Shlomo Geva. 1995. RULEX & CEBP Networks As the Basis for a Rule Refinement System. In Hybrid Problems, Hybrid Solutions , J. Hallam (Ed.). IOS Press, 1–12

1995