Pith. sign in

REVIEW 4 major objections 5 minor 46 references

Deriving Equivalent Symbol-Based Decision Models from Feedforward Neural Networks

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Trained feedforward nets can be rewritten as transparent decision trees.

desk verdict The prototype is real but the equivalence claim is not established: the method traces finite input paths with an undefined pruning threshold, so the title promise fails as stated. read the letter →

arxiv 2504.12446 v2 pith:YQVG2SCA submitted 2025-04-16 cs.LG cs.AI

Sebastian Seidel , Uwe M. Borghoff This is my paper · ORCID
classification cs.LGcs.AI
keywords ArtificialneuralnetworksconnectionismdecisiontreesexplainableAIsymbolicmodelssymbolismfeedforwardmodelinterpretability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a trained feedforward neural network, including convolutional networks expressed in feedforward form, can be converted into a decision tree that reproduces the network's decisions and exposes how each decision was reached. The conversion works by tracing, from the output layer backward, which weighted connections actually matter for a given input, and representing the surviving input-to-neuron combinations as symbolic decisions along a tree path. The authors claim the resulting tree effectively captures the network's decision process and that the procedure scales to deeper networks by replacing each relevant edge with a subpath derived from the preceding hidden layer. If the claim holds, trained networks would gain a transparent, human-readable audit trail without retraining or altering the network itself.

What carries the argument

The central machinery is the filler/role decomposition borrowed from tensor-product symbol representations: a symbol is written $s = \sum_i f_i \otimes r_i$, where $f_i$ is the filler and $r_i$ its role. In the network, each input neuron's activation value is a filler, the information it encodes is its role, and a neuron's input sum $\sum_i v_i w_i$ is the combination rule that decides which (filler, role) tuples survive. The second load-bearing component is the recursive relevance cut: an edge is kept only if the product $v_j \times w_{jk}$ is sufficiently large for the receiving neuron's activation, and this criterion is applied backward from the output layer to every hidden layer, producing subpaths that replace edges of an already-built path. Iterating this for many input vectors and merging the resulting paths yields the hierarchical decision tree.

What would settle it

Take a trained classifier, pick an input whose tree path is built with a given relevance cutoff, then search for a perturbation that only alters connections below the cutoff or inputs that change their weighted contributions, and watch whether the network's argmax changes while the extracted tree's decision stays fixed. Any single such divergence demonstrates the extracted tree is not equivalent to the network.

Watch

Extended reading notes

Core claim

The paper's central claim is that a feedforward neural network's decision for a given input vector can be re-expressed as a decision path, and that the union of such paths over inputs forms a hierarchical decision tree that is functionally equivalent to the network. The derivation identifies the symbolic components of the network's distributed representation: input values play the role of fillers, their positions and the information they encode play the role of roles, and each neuron's weighted input sum is the combination rule that binds them. Starting from the output layer, the method keeps only neurons and connections whose weighted activation is sufficiently large to influence the output, then repeats this relevance check layer by layer so that every surviving edge of the tree is expanded into a subpath from the previous hidden layer. The result is presented as a step-by-step algorithmic procedure, with a prototype that reads a stored model from an .h5 file, converts convolutional and pooling layers into feedforward form, and renders the merged decision tree in a graphical interface.

Load-bearing premise

The extraction assumes that weighted connections whose product $v_j \times w_{jk}$ is not judged sufficiently large can be discarded without ever changing the network's final decision; if small contributions collectively flip the output layer's argmax, the extracted tree diverges from the network.

Editorial extensions

If this is right

  • Every input decision of a feedforward network can be given a concrete branch-by-branch explanation in the vocabulary of the original input features.
  • The extraction runs on a stored trained model and produces a static tree, so the audit object is independent of the training framework.
  • Because convolutional layers are first flattened into feedforward form, the same derivation covers convolutional classifiers, not only fully connected networks.
  • The method is modular per hidden layer: adding depth is handled by replacing edges with subpaths, so deeper networks do not require a new algorithm.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the relevance threshold is made adaptive or formally tied to the output margin, the method could double as a verification tool: the gap between the pruned tree and the true network output measures fidelity, and a user could demand whatever fidelity the application requires.
  • The same filler/role reading could produce other symbolic artifacts, such as rules or state machines, from the same extracted paths, potentially connecting this procedure to broader neuro-symbolic compilation.
  • A natural stress test is to compare tree and network on adversarial inputs engineered near decision boundaries, since these are exactly the cases where pruned small contributions are most likely to flip the argmax.
  • The paper's equivalence is per input-vector derivation; whether the merged tree is equivalent on unseen inputs is an empirical question that the authors do not address.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a method for deriving decision trees from trained feedforward neural networks (FNNs), using the filler/role distinction from Smolensky's integrated connectionist/symbolic architecture to map neuron activations and weighted connections to decision tree edges and nodes. The derivation proceeds per input vector: activations are computed, 'relevant' weighted connections are selected by an informally defined threshold, and the resulting decision paths are merged into a hierarchical tree. A Java prototype reads Keras .h5 files, converts convolutional and pooling layers into feedforward form, and visualizes the extracted tree. The central claim, stated in the abstract and Section V, is that the resulting symbolic structures are 'equivalent' to the network and 'preserve its functionality', with scalability to deeper networks via iterative subpath refinement.

Significance. If the equivalence claim were established, this would be a valuable contribution to explainable AI, offering a transparent, faithful symbolic surrogate for FNN decisions. The paper's strengths include a concrete step-by-step procedure, illustrated examples, explicit handling of convolutional and pooling layers, and a publicly available prototype. However, the central notion of equivalence is not formally defined or proven, and the method relies on an unspecified relevance threshold that can affect fidelity. The paper provides no quantitative validation that extracted trees reproduce network outputs on unseen inputs. As presented, the contribution is closer to a heuristic, per-instance explanation tool than to an equivalence-preserving transformation, so the stated significance is not yet supported.

major comments (4)
  1. [Abstract; Section V] The central claim of equivalence is not established. Section V states that for 'each of the input vectors to be examined' a decision path is derived and then combined into a tree; there is no argument that the finite tree obtained from a finite set of traced inputs reproduces the network's decision function on inputs outside that set. Since the input domain is continuous, a coverage or generalization argument is required to justify the term 'equivalent'. Without it, the method at best provides per-instance explanations, not an equivalent symbolic model.
  2. [Section III.D, steps 1-3; Section III.E, steps 1-2] The pruning criterion is undefined. The method repeatedly discards weighted connections whose product v_j*w_j is not 'sufficiently large' or 'sufficiently relevant', but no threshold, justification, or error bound is given. A connection with a small individual contribution can be decisive for the output-layer argmax, especially when contributions accumulate; thus the pruned tree may disagree with the original network even on the traced input. The paper needs to specify how the threshold is chosen and to demonstrate, at minimum empirically, that the choice does not alter the final decision across a representative set of inputs.
  3. [Section IV; Section V] No quantitative validation is provided. Section IV describes a prototype and Section V concludes that the prototype 'proved that the desired derivation of equivalent symbol-based decision models from feedforward neural networks can be efficiently implemented.' This only demonstrates implementation feasibility. The paper reports no experiments comparing the extracted decision tree's outputs with the original network on held-out data, no accuracy/fidelity metrics, and no sensitivity analysis for the relevance threshold. Without such measurements, the central claim of functionality preservation is unsubstantiated.
  4. [Abstract; Section III.E] The scalability claim to deeper networks is not supported. The abstract states that the method 'enable[s] scalability to deeper networks through iterative refinement of subpaths for each hidden layer', but Section III.E simply repeats the same undefined pruning step per layer. Recursively applying a heuristic that lacks an error bound does not transfer any equivalence guarantee to deeper networks. The paper provides no complexity analysis and no demonstration that iterative subpath refinement preserves the network's output mapping.
minor comments (5)
  1. [Section III.D] There are typographical errors in the running example: 'by is not considered' should likely be 'bv is not considered', and 'bu' is used without being defined as the input value of the first input neuron; the notation should be made consistent.
  2. [Throughout] The text contains odd spacing artifacts such as 'K ERAS', 'T ENSOR FLOW', and 'J AVA'; these should be corrected to the standard software names during copyediting.
  3. [Section III.C] The formal definition of a decision tree is difficult to parse; for example, the notation s×t×b_ijr → e_r and the statement 'm = number of initial edges' are ambiguous. Please clarify with a cleaner formalization, or rely on a standard reference.
  4. [Section II] The related-work discussion is very broad and does not compare the proposed extraction method with existing decompositional approaches (e.g., DeepRED, LORE, or LRP-based methods). Adding such a comparison would help position the contribution.
  5. [Section IV] The prototype description focuses on implementation details (HDF5 parsing, JavaFX GUI, layer conversion) but does not describe how large networks perform; a brief runtime or scalability experiment would strengthen the claims in Section V.

Circularity Check

2 steps flagged · score 6.0 of 10

The claimed equivalence is tautological: each decision path is built from the same input vector's activations and from relevance scores defined by that vector's already-computed output, so the tree's agreement on traced inputs holds by construction rather than by independent verification.

  1. self definitional [Section III.D, steps 1-3 (Identification of symbols and their relationships)]
    "if a neuron j has no connection to a neuron k in the output layer through which a sufficiently large weighted value vjk× wjk that is significant for the calculated output activation, then the neuron j is not considered for the decision path in a decision tree."

    The criterion for including a neuron or input edge in the derived decision path is defined in terms of 'the calculated output activation'—the very decision the path is supposed to reproduce. The path is therefore selected using the target output, so the path's connection to that output is a consequence of the selection rule, not an independent finding. 'Sufficiently large' and 'sufficiently relevant' are never given numerical thresholds or error bounds in Sections III.D-E, leaving 'contributed to this calculated output' as the only concrete content of the relevance predicate. Recursively applying this rule in deeper layers cannot convert a definitional selection into an equivalence proof for unseen inputs.

  2. fitted input called prediction [Section V (Summary and Conclusion), step list for deriving hierarchical decision paths]
    "For each of the input vectors to be examined, the following steps are performed: 1) the corresponding input vector is entered into the network and the resulting activations of all neurons for this input vector are calculated based on the minimized structure. 2) based on the calculated activations instead of the estimated boundaries, the relevant weighted connections and neurons of the minimized structure are calculated for the classification of the selected input vector."

    Each hierarchical decision path is derived from the activations produced by the same input vector whose classification the path is said to reproduce, and the 'relevant' connections are chosen using that vector's computed output. Consequently, for every input in the traced set, the path must terminate at the network's own decision: the match is guaranteed by construction. Combining such per-input paths into a tree does not establish that the tree is an 'equivalent symbol-based decision model' over the continuous input domain, since no coverage argument or equivalence proof is supplied. The claimed preservation of functionality for the traced inputs is thus a restatement of the tracing procedure rather than a derived predictive result.

full rationale

The paper's method is a white-box, per-input activation trace: the FNN is run on an input, and the path is built from the neurons and weighted connections deemed relevant to that run. For a single input, this can be a legitimate explanation of the network's internal processing, and the paper's prototype does demonstrate that such traces can be assembled into a tree structure. The circularity enters where the paper upgrades the trace into a global 'equivalent symbol-based decision model' that preserves the FNN's functionality. The tree is constructed from the very inputs and outputs it is claimed to reproduce; no held-out inputs, no out-of-distribution tests, and no coverage proof are provided. Section V's statement that 'This proved that the desired derivation of equivalent symbol-based decision models from feedforward neural networks can be efficiently implemented' reports only that the procedure runs, not that the resulting tree matches the network on unseen inputs. The relevance threshold is undefined, so the only operational meaning of 'relevant' is 'significant for the calculated output activation' (Section III.D), making the path's correspondence to that output definitional. Even on traced inputs, dropping a supposedly insufficient connection can change an output-layer argmax, so the equivalence claim is neither proven nor tautologically safe for all inputs; it rests on an unspecified heuristic. The self-citations in the reference list ([16], [39]) are background on symbolic/connectionist tradeoffs and prior extraction work and are not the load-bearing step here, so no separate self-citation circularity is charged. Overall, the central equivalence claim reduces to a per-input fit by construction, while the algorithm retains some independent content as a way to generate per-decision explanation paths; this warrants a partial circularity score of 6 rather than a full 8-10. The correct fix would be an explicit error bound on the pruning threshold, a proof that excluded connections cannot alter the output-layer argmax, and an evaluation against held-out inputs.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on an unproven pruning assumption (small weighted activations can be ignored) and on the assumption that a continuous neural decision can be captured by a discrete tree. The only explicit free parameter is the relevance threshold, which is never specified. No new physical or scientific entities are introduced.

free parameters (1)
  • relevance threshold for weighted activations = unspecified
    Section III.D and III.E repeatedly require that weighted activations be sufficiently large or sufficiently relevant to be included in a decision path, but no numerical threshold or statistical criterion is given. The cutoff is a hand-set parameter that determines which connections and neurons appear in the extracted tree.
assumptions (4)
  • domain assumption A trained feedforward neural network with continuous activations can be represented by a finite decision tree over discretized input configurations without loss of decision accuracy.
    The entire method presupposes that symbolic, discrete branching can capture the continuous feedforward computation. This is asserted in Section III.C and Section III.D and never proven.
  • ad hoc to paper Dropping connections with small weighted activations does not alter the final output decision.
    Introduced in Section III.D, steps 1 to 3, and applied recursively in Section III.E. No justification is given beyond illustrative examples, and small contributions could in principle change the output layer's argmax.
  • domain assumption Neurons in the same layer can be ordered arbitrarily to define the sequence of tests in the decision path.
    Section III.D states that the arrangement of the neurons of the penultimate layer is arbitrary. This imposes a total order on tests that may not correspond to the network's actual parallel processing.
  • standard math Convolutional and pooling layers can be exactly converted to feedforward form with sparse weights and appropriate padding and strides.
    Section IV.A and IV.B use this standard equivalence to convert CNN layers into feedforward form for the prototype. This is a standard result and is not the main weakness.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Deriving Equivalent Symbol-Based Decision Models from Feedforward Neural Networks." pith.science (2026). https://pith.science/paper/YQVG2SCA

@misc{pith2026250412446,
  author       = {Pith},
  title        = {Pith review of: Deriving Equivalent Symbol-Based Decision Models from Feedforward Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YQVG2SCA}},
  note         = {Machine review of arXiv:2504.12446}
}
read the original abstract

Artificial intelligence (AI) has emerged as a transformative force across industries, driven by advances in deep learning and natural language processing, and fueled by large-scale data and computing resources. Despite its rapid adoption, the opacity of AI systems poses significant challenges to trust and acceptance. This work explores the intersection of connectionist and symbolic approaches to artificial intelligence, focusing on the derivation of interpretable symbolic models, such as decision trees, from feedforward neural networks (FNNs). Decision trees provide a transparent framework for elucidating the operations of neural networks while preserving their functionality. The derivation is presented in a step-by-step approach and illustrated with several examples. A systematic methodology is proposed to bridge neural and symbolic paradigms by exploiting distributed representations in FNNs to identify symbolic components, including fillers, roles, and their interrelationships. The process traces neuron activation values and input configurations across network layers, mapping activations and their underlying inputs to decision tree edges. The resulting symbolic structures effectively capture FNN decision processes and enable scalability to deeper networks through iterative refinement of subpaths for each hidden layer. To validate the theoretical framework, a prototype was developed using Keras .h5-data and emulating TensorFlow within the Java JDK/JavaFX environment. This prototype demonstrates the feasibility of extracting symbolic representations from neural networks, enhancing trust in AI systems, and promoting accountability.

Figures

Figures reproduced from arXiv: 2504.12446 by the authors.

Figure 1
Figure 1. Symbolic vs. connectionist approaches. of the determined characteristics in the recognized problem context, and finally the desired solution is displayed. Perhaps the best known example of this category of AI systems are flat artificial neural networks, which are based on the idea of the perceptron first described in [22]. As Goodfellow et al. [18] explain, connectionist approaches were called cybernetics from 1940 … view at source ↗
Figure 2
Figure 2. ANNs’ learning approaches and the problem categories in focus [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Training-time dependent derivation of equivalent symbol-based [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (16 more)
Figure 4
Figure 4. Figure 4: Basic principles of parallel information processing in ANNs with hidden cells. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Example of the concept of fillers and the roles in the integrated [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Intended use of the concepts of fillers, roles and combinations of [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: A concrete realization of a decision tree that corresponds to the [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: Example of a formalized decision tree. are performed, correspond to the principle of possible roles to be assumed. Together with an associated filler from an as￾signment, they form a complete symbol. A defined connecting edge er is always connected to this symbol {i0, …
Figure 9
Figure 9. Figure 9: Exemplary representation of the elements of a single path of a decision tree that can be derived from a single run of a given feedforward neural [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: Exemplary visualization of the principle of dependence of individual [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]
Figure 11
Figure 11. Figure 11: Exemplary illustration of the concept of integrating distributed representation and processing of feedforward neural networks into extended decision [PITH_FULL_IMAGE:figures/full_fig_p010_11.png]
Figure 12
Figure 12. Figure 12: Exemplary illustration of the expansion of decision paths in extended decision trees for feedforward neural networks with several intermediate [PITH_FULL_IMAGE:figures/full_fig_p010_12.png]
Figure 13
Figure 13. Figure 13: Description of the special features of a convolution using a kernel compared to hidden cells based on the descriptions in [ [PITH_FULL_IMAGE:figures/full_fig_p011_13.png]
Figure 14
Figure 14. Figure 14: Example of a Conv2D layer processing a 2D input with given strides [2,1] and valid padding. A. The layers In KERAS, the input layer is defined as an object of type Input, as described in [43] under Layers API / Core layers / Input object. This object represents the in…
Figure 15
Figure 15. Figure 15: Example of the transfer of a 2D convolution into the filter of a feedforward layer using same padding. [PITH_FULL_IMAGE:figures/full_fig_p012_15.png]
Figure 18
Figure 18. Figure 18: Visualization of the data structure behind the [PITH_FULL_IMAGE:figures/full_fig_p013_18.png]
Figure 16
Figure 16. Figure 16: Visualization of the implementation of the principle of flattening [PITH_FULL_IMAGE:figures/full_fig_p013_16.png]
Figure 17
Figure 17. Figure 17: Representation of the Merger class, including its associated methods. D. The graphical user interface The structure of the artificial neural network to be analyzed is loaded from the associated .h5-file by the main-class each time the prototype is started. After loadi…
Figure 19
Figure 19. Figure 19: The start screen of the graphical user interface of the prototype after starting the program and generating the feedforward form of a read-in artificial [PITH_FULL_IMAGE:figures/full_fig_p014_19.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 33 canonical work pages

  1. [16]

    Understanding neural network decisions by creating equivalent symbolic AI models,

    S. Seidel, S. Schimmler, and U. M. Borghoff, “Understanding neural network decisions by creating equivalent symbolic AI models,” in Intelligent Systems and Applications - Proceedings of the 2018 Intelligent Systems Conference, IntelliSys 2018, London, UK, September 6-7, 2018, Volume 1 , ser. Advances in Intelligent Systems and Computing, K. Arai, S. Kapoo...

    work page doi:10.1007/978-3-030-01054-6 2018

  2. [1]

    A review of trustworthy and explainable artificial intelligence (XAI),

    V . Chamola et al. , “A review of trustworthy and explainable artificial intelligence (XAI),” IEEE Access , vol. 11, pp. 78 994–79 015, 2023. [Online]. Available: https://doi.org/10.1109/ACCESS.2023.3294569

    arXiv 2023

  3. [2]

    Peeking inside the black-box: A survey on explainable artificial intelligence (XAI),

    A. Adadi and M. Berrada, “Peeking inside the black-box: A survey on explainable artificial intelligence (XAI),” IEEE Access , vol. 6, pp. 52 138–52 160, 2018. [Online]. Available: https://doi.org/10.1109/ ACCESS.2018.2870052

    arXiv 2018

  4. [3]

    Explaining explanations: An overview of interpretability of machine learning,

    L. H. Gilpin, D. Bau, B. Z. Yuan, A. Bajwa, M. A. Specter, and L. Kagal, “Explaining explanations: An overview of interpretability of machine learning,” in 5th IEEE International Conference on Data Science and Advanced Analytics, DSAA 2018, Turin, Italy, October 1-3, 2018, F. Bonchi, F. J. Provost, T. Eliassi-Rad, W. Wang, C. Cattuto, and R. Ghani, Eds. I...

    arXiv 2018

  5. [4]

    On the evaluation of the symbolic knowledge extracted from black boxes,

    F. Sabbatini and R. Calegari, “On the evaluation of the symbolic knowledge extracted from black boxes,” AI Ethics , vol. 4, no. 1, pp. 65–74, 2024. [Online]. Available: https://doi.org/10.1007/s43681-023-00406-1 DERIVING EQUIV ALENT SYMBOL-BASED DECISION MODELS FROM FEEDFORW ARD NEURAL NETWORKS 15

    work page doi:10.1007/s43681-023-00406-1 2024

  6. [5]

    The application of named entity recognition in military intelligence,

    C. Nitzl, A. Cyran, S. Krstanovic, and U. M. Borghoff, “The application of named entity recognition in military intelligence,” in Computer Aided Systems Theory - EUROCAST 2024 - 19th International Conference, Las Palmas de Gran Canaria, Spain, February 25 - March 1, 2024, Revised Selected Papers, Part I, ser. Lecture Notes in Computer Science, A. Q. Arenc...

    work page 2024

  7. [6]

    The Use of Artificial Intelligence in Military Intelligence: An Experimental Investigation of Added Value in the Analysis Process

    C. Nitzl, A. Cyran, S. Krstanovic, and U. M. Borghoff, “The use of artificial intelligence in military intelligence: An experimental inves- tigation of added value in the analysis process,” arXiv Preprint , vol. abs-2412.03610, pp. 1–28, 2024. [Online]. Available: https://doi.org/10. 48550/arXiv.2412.03610

    work page Pith review arXiv doi:10.48550/arxiv.2412.03610 2024

  8. [7]

    Artificial intelligence crime: An overview of malicious use and abuse of AI,

    T. F. Blauth, O. J. Gstrein, and A. J. Zwitter, “Artificial intelligence crime: An overview of malicious use and abuse of AI,” IEEE Access, vol. 10, pp. 77 110–77 122, 2022. [Online]. Available: https://doi.org/10.1109/ACCESS.2022.3191790

    arXiv 2022

Show all 46 references
  1. [8]

    A survey on neural network interpretability,

    Y . Zhang, P. Ti ˜no, A. Leonardis, and K. Tang, “A survey on neural network interpretability,” IEEE Trans. Emerg. Top. Comput. Intell., vol. 5, no. 5, pp. 726–742, 2021. [Online]. Available: https://doi.org/10.1109/TETCI.2021.3100641

    2021

  2. [9]

    Feed-forward neural networks,

    G. Bebis and M. Georgiopoulos, “Feed-forward neural networks,” IEEE Potentials, vol. 13, no. 4, pp. 27–31, 1994

    1994

  3. [10]

    Explainable AI (XAI): core ideas, techniques, and solutions,

    R. Dwivedi et al. , “Explainable AI (XAI): core ideas, techniques, and solutions,” ACM Comput. Surv., vol. 55, no. 9, pp. 194:1–194:33, 2023. [Online]. Available: https://doi.org/10.1145/3561048

    2023 doi

  4. [11]

    Recent advances in decision trees: an updated survey,

    V . G. Costa and C. E. Pedreira, “Recent advances in decision trees: an updated survey,” Artif. Intell. Rev. , vol. 56, no. 5, pp. 4765–4800,

  5. [12]

    Russel and P

    S. Russel and P. Norvig, Artificial Intelligence – A Modern Approach . Prentice Hall, 2010

    2010

  6. [13]

    Logical vs. analogical or symbolic vs. connectionist or neat vs. scruffy,

    M. Minsky, “Logical vs. analogical or symbolic vs. connectionist or neat vs. scruffy,” in Artificial Intelligence at MIT: Expanding Frontiers ,

  7. [14]

    Abadi et al

    M. Abadi et al. , TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems - API , 2015, version 2.5.1. [Online]. Available: https://www.tensorflow.org/api docs/python/tf

    2015

  8. [15]

    A novel internet of things web attack detection architecture based on the combination of symbolism and connectionism AI,

    Y . An et al. , “A novel internet of things web attack detection architecture based on the combination of symbolism and connectionism AI,” IEEE Internet Things J. , vol. 11, no. 11, pp. 19 823–19 837, 2024. [Online]. Available: https://doi.org/10.1109/JIOT.2024.3369852

    2024

  9. [17]

    Deep learning in neural networks: An overview,

    J. Schmidhuber, “Deep learning in neural networks: An overview,” CoRR, vol. abs/1404.7828, 2014. [Online]. Available: http://arxiv.org/ abs/1404.7828

    2014 arXiv

  10. [18]

    Goodfellow, Y

    I. Goodfellow, Y . Bengio, and A. Courville, Deep Learning. The MIT Press, 2016

    2016

  11. [19]

    Connectionist inference models,

    A. Browne and R. Sun, “Connectionist inference models,” Neural Networks, vol. 14, no. 10, pp. 1331–1355, 2001. [Online]. Available: https://doi.org/10.1016/S0893-6080(01)00109-5

    2001 doi

  12. [20]

    Connectionist model generation: A first-order approach,

    S. Bader, P. Hitzler, and S. H ¨olldobler, “Connectionist model generation: A first-order approach,” Neurocomputing, vol. 71, no. 13-15, pp. 2420– 2432, 2008. [Online]. Available: https://doi.org/10.1016/j.neucom.2007. 10.028

    2008 doi

  13. [21]

    G ´eron, Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow

    A. G ´eron, Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow. ”O’Reilly Media, Inc.”, 2022

    2022

  14. [22]

    The perceptron: A probabilistic model for information storage and organization in the brain,

    F. Rosenblatt, “The perceptron: A probabilistic model for information storage and organization in the brain,” Psychological Review, pp. 65– 386, 1958

    1958

  15. [23]

    Toward human-understandable, explainable AI,

    H. Hagras, “Toward human-understandable, explainable AI,” Computer, vol. 51, no. 9, pp. 28–36, 2018. [Online]. Available: https: //doi.org/10.1109/MC.2018.3620965

    2018

  16. [24]

    Recent advances in trustworthy explainable artificial intelligence: Status, challenges, and perspectives,

    A. Rawal, J. McCoy, D. B. Rawat, B. M. Sadler, and R. S. Amant, “Recent advances in trustworthy explainable artificial intelligence: Status, challenges, and perspectives,” IEEE Trans. Artif. Intell. , vol. 3, no. 6, pp. 852–866, 2022. [Online]. Available: https: //doi.org/10.1...

    2022

  17. [25]

    Four principles of explainable artificial intelligence,

    P. J. Phillips, C. A. Hahn, P. C. Fontana, A. N. Yates, K. Greene, D. A. Broniatowski, and M. A. Przybocki, “Four principles of explainable artificial intelligence,” NIST Interagency/Internal Report (NISTIR), 2021. [Online]. Available: https://nvlpubs.nist.gov/nistpubs/ ir/202...

    2021

  18. [26]

    U. S. government, Executive Order on AI , 2023. [Online]. Available: https://ai.gov/actions/

    2023

  19. [27]

    N. I. of Standards and Technology. (2023) AI risk management framework. [Online]. Available: https://nvlpubs.nist.gov/nistpubs/ai/ NIST.AI.100-1.pdf

    2023

  20. [28]

    XAIR: A systematic metareview of explainable AI (XAI) aligned to the software development process,

    T. Clement, N. Kemmerzell, M. Abdelaal, and M. Amberg, “XAIR: A systematic metareview of explainable AI (XAI) aligned to the software development process,” Machine Learning and Knowledge Extraction, vol. 5, no. 1, pp. 78–108, 2023. [Online]. Available: https://www.mdpi.com/250...

    2023

  21. [29]

    Turek, Explainable Artificial Intelligence

    M. Turek, Explainable Artificial Intelligence . The U.S. Defense Advanced Research Projects Agency. [Online]. Available: https: //www.darpa.mil/program/explainable-artificial-intelligence

  22. [30]

    D. E. Rumelhart and J. L. McClelland, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 1: Foundations. Boston: The MIT Press, 1986

    1986

  23. [31]

    Unsupervised learning based on artificial neural network: A review,

    H. U. Dike, Y . Zhou, K. K. Deveerasetty, and Q. Wu, “Unsupervised learning based on artificial neural network: A review,” in 2018 IEEE International Conference on Cyborg and Bionic Systems (CBS) , 2018, pp. 322–327. [Online]. Available: https://doi.org/10.1109/CBS.2018. 8612259

    2018 doi

  24. [32]

    Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex,

    D. H. Hubel and T. N. Wiesel, “Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex,” The Journal of Physiology, vol. 160, 1962

    1962

  25. [33]

    Neocognitron: A neural network model for a mechanism of visual pattern recognition,

    K. Fukushima, S. Miyake, and T. Ito, “Neocognitron: A neural network model for a mechanism of visual pattern recognition,” IEEE Transactions on Systems, Man, and Cybernetics , vol. SMC-13, pp. 826–834, 1983. [Online]. Available: https://link.springer.com/article/10. 1007/BF00344251

    1983

  26. [34]

    On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation,

    S. Bach, A. Binder, G. Montavon, F. Klauschen, K.-R. M ¨uller, and W. Samek, “On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation,” PLoS One , 2015. [Online]. Available: https://pubmed.ncbi.nlm.nih.gov/26161953/

    2015

  27. [35]

    van Veen, The Neural Network Zoo , 2016

    F. van Veen, The Neural Network Zoo , 2016. [Online]. Available: https://www.asimovinstitute.org/neural-network-zoo/

    2016

  28. [36]

    Smolensky and G

    P. Smolensky and G. Legendre, The harmonic mind: From neural computation to optimality-theoretic grammar (Cognitive architecture), Vol. 1. MIT press, 2006

    2006

  29. [37]

    Millington and J

    I. Millington and J. Funge, Artificial Intelligence for Games - Second Edition. Morgan Kaufmann Publishers, 2009

    2009

  30. [38]

    Decision trees,

    L. Rokach and O. Maimon, “Decision trees,” The Data Mining and Knowledge Discovery Handbook , vol. 6, 2005

    2005

  31. [39]

    Extracting state machines from feedforward neural networks,

    S. Seidel and U. M. Borghoff, “Extracting state machines from feedforward neural networks,” in 9th IEEE International Conference on Cognitive Infocommunications, CogInfoCom 2018, Budapest, Hungary, August 22-24, 2018 . IEEE, 2018, pp. 11–16. [Online]. Available: https://doi.or...

    2018

  32. [40]

    [Online]

    HDFGroup, Introduction to HDF5 , 2006, version 2016. [Online]. Available: https://portal.hdfgroup.org/display/HDF5

    2006

  33. [41]

    [Online]

    JSON.org, Einf¨uhrung in JSON , 2017, version 2017. [Online]. Available: https://www.json.org/json-de.html

    2017

  34. [42]

    Mudd, jHDF - Pure Java HDF5 library , 2015, version 2020

    J. Mudd, jHDF - Pure Java HDF5 library , 2015, version 2020. [Online]. Available: https://github.com/jamesmudd/jhdf

    2015

  35. [43]

    Chollet et al

    F. Chollet et al. , Keras, 2015, version 2.4.0. [Online]. Available: https://keras.io/api/

    2015

  36. [44]

    Seidel, IntelliJ Project, 2025

    S. Seidel, IntelliJ Project, 2025. [Online]. Available: https://github.com/ SeSeidel/CalculatorDemonstrator/tree/main

    2025

  37. [1991]

    Available: https://redirect.cs.umbc.edu/courses/471/ papers/minksy91.pdf

    [Online]. Available: https://redirect.cs.umbc.edu/courses/471/ papers/minksy91.pdf

  38. [2023]

    Available: https://doi.org/10.1007/s10462-022-10275-5

    [Online]. Available: https://doi.org/10.1007/s10462-022-10275-5

    doi

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.