Pith. sign in

REVIEW 4 major objections 5 minor 24 references

Enhancing Symbolic Machine Learning by Subsymbolic Representations

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper establishes that equipping the symbolic rule learner TILDE with a similarity predicate over pretrained neural embeddings, followed by fine-tuning those embeddings with a logic tensor network, yields the best F1 score among all…

desk verdict A sensible incremental neurosymbolic trick—similarity predicates over pretrained embeddings plus LTN fine-tuning—that shows a consistent F1 trend but is oversold relative to a single-split evaluation with no external baselines. read the letter →

arxiv 2506.14569 v1 pith:QP22OJGG submitted 2025-06-17 cs.AI cs.LO

Stephen Roth , Lennart Baur , Derian Boer , Stefan Kramer This is my paper · ORCID
classification cs.AIcs.LO
keywords neuro-symbolicAIinductivelogicprogrammingTILDEneuralembeddingssimilaritypredicateTensorNetworksembeddingfine-tuningF1score
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's aim is to give a symbolic machine learning system access to neural embeddings without giving up its interpretability. It introduces a similarity predicate over the embeddings, lets the symbolic learner use it as background knowledge, and then refines the embeddings so the learned rules are better satisfied. In three classification tasks the complete pipeline beats plain symbolic learning and hand-crafted rules in F1.

What carries the argument

The central object is the predicate similar/2, defined by cosine similarity: similar(X,Y) holds when the dot product of the embeddings of X and Y, divided by the product of their norms, is at least a threshold tau. TILDE is biased to use this predicate with a constant on one side, so it learns words or genes that act as semantic prototypes. The accompanying logic tensor network converts the induced tree into fuzzy rules and backpropagates through the cosine-similarity predicate, moving embeddings so that the rules they trigger match the labels; the updated embeddings then replace the originals for classification.

What would settle it

Replacing the pretrained word and gene vectors with random vectors of equal dimension, while keeping all thresholds and rules fixed, would falsify the semantic-similarity claim if the F1 gains persisted.

Watch

Extended reading notes

Core claim

The central discovery is that a purely relational learner can be lifted by a single cosine-similarity predicate. The paper reports that TILDE with the predicate raises F1 from 0.257 to 0.480 on hate speech, from 0.574 to 0.626 on spam, and from 0.167 to 0.242 on drug response. Fine-tuning all embeddings through a logic tensor network pushes these numbers to 0.576, 0.763, and 0.295, which the authors state is the best among all variants and baselines. This claim concerns F1 specifically: accuracy gains are mixed in the reported table.

Load-bearing premise

The approach succeeds only if cosine similarity over the pretrained embeddings is a meaningful relation for the target, with the per-dataset threshold tau chosen by hand and no sensitivity analysis reported.

Editorial extensions

If this is right

  • In all three domains, adding the similarity predicate alone improves F1 over plain TILDE, so the gain does not require the LTN refinement step.
  • Fine-tuning all embeddings, rather than only the constants appearing in the rules, gives the largest F1 gains, though it moves embeddings further from their original semantic positions.
  • Fine-tuning can repair poor symbolic choices: in spam, the embedding for 'urgent' shifts toward 'text', which TILDE independently considered the most predictive constant.
  • Because the final classifier is still a TILDE tree, the improved F1 comes with readable rules rather than a black box.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One testable extension: sweep the similarity threshold tau on a single dataset; if the reported ordering of methods changes drastically, the fixed thresholds carry more weight than the paper's single-setting comparison shows.
  • The same wrapper could be applied to other symbolic learners or to similarities between whole instances, effectively implementing kernels inside a logical language; the paper lists this as future work, and it follows from the mechanism rather than from the experiments.
  • The authors' own data-provenance note leaves open a residual risk that public text embeddings overlapped with target texts during pretraining; a cleaner test would use embeddings trained without the evaluation corpus.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a neuro-symbolic classification pipeline that augments the ILP system TILDE with a similar/2 predicate computed from cosine similarity over pretrained embeddings (GloVe for text, GenePT for genes). TILDE induces a logical decision tree that can use this predicate to generalize beyond exact lexical matches. The tree is then converted into fuzzy rules for a Logic Tensor Network, which fine-tunes the embeddings to increase satisfaction of the rules; the refined embeddings are fed back into the original TILDE tree for classification. The approach is evaluated on hate-speech, SMS spam, and drug-response datasets. Table 1 reports accuracy and F1 for TILDE without embeddings, TILDE with similar/2, TILDE with LTN fine-tuning of constants or all embeddings, and for hand-crafted rules with and without LTN fine-tuning. The authors claim that the maximal variant, TILDE with LTN-revised all embeddings, outperforms all other variants and baseline methods in terms of F1 score across all three domains.

Significance. If the empirical claim were established, the paper would make a useful and modular contribution: a lightweight way to inject subsymbolic information into symbolic rule learning while retaining interpretable decision trees. The paper's strengths include a clear pipeline, the use of public datasets and standard pretrained embeddings, and a useful ablation that separates the contribution of the similar/2 predicate from the LTN fine-tuning step. The explicit discussion of data provenance is also a positive feature. However, the headline superiority claim is not currently supported by the reported evaluation: the evidence consists of single-split F1 values with no variance estimates, no significance tests, and no comparison to standard classifiers. The central idea is plausible and interesting, but the experimental protocol needs substantial strengthening before the claim can be accepted.

major comments (4)
  1. [Section 4.2, Table 1] The headline claim that TILDE+LTN all 'outperforms all other variants' and 'all other baseline methods' is not supported by the reported experimental protocol. The results come from a single train/test split, with no standard deviations, confidence intervals, or significance tests. On the hate-speech task, the margin over TILDE+LTN constants is only 0.576 versus 0.569, and over Hand-crafted+LTN all it is 0.576 versus 0.561, on a test set of roughly 500 instances; these differences are within the range of sampling noise. Please add repeated runs with different seeds, bootstrap confidence intervals, and/or a paired significance test such as McNemar's test, and report the main effect sizes with uncertainty.
  2. [Section 4.1, Table 1] The comparison set contains only TILDE variants and hand-crafted rules. The abstract's statement that the approach 'outperforms all other baseline methods' is therefore an overstatement: no standard supervised baselines such as logistic regression, SVM, random forest, or a simple neural model are included. Such baselines are needed to calibrate whether the reported F1 values are competitive in absolute terms. Please add at least one or two standard classifiers per dataset, ideally with the same or comparable input features.
  3. [Algorithm 1, line 13] The right-branch rule construction appears to negate the entire accumulated conjunction rather than only the current node test. Specifically, rule+ is defined as rulecomp union Conjunction(node), and the right child receives rulecomp union Not(QuantifyAndConjunct(rule+)). This makes the right-branch body logically equivalent to 'not(previous tests and current test)', which holds when earlier tests on the path are false, so the resulting rule can fire in situations that cannot reach the right branch in the original decision tree. The correct right-branch body should be rulecomp plus the negation of the current test, with quantifiers handled accordingly. Please correct the pseudocode, or explain why the current formulation is intended, and confirm that the experiments use the corrected semantics. This point is load-bearing because the LTN fine-tuning signal is derived from these rules.
  4. [Section 3.2 and Section 4.1] The similar/2 threshold tau is a dataset-specific free parameter (0.75 for hate speech, 0.70 for spam, 0.50 for drug response), and no sensitivity analysis is reported. Since the performance gains are attributed to the similarity predicate, the results should be shown to be robust to tau (e.g., a small grid around the chosen values), or the threshold should be learned as part of the pipeline. Without this, the reader cannot tell whether the reported improvements depend on hand-tuned thresholds that may not transfer to new data.
minor comments (5)
  1. [Throughout] There are several typos: 'sytems' in Section 1, 'versality' in Section 1, 'Kontschieder el al.' in Section 2, 'simlilar' in Section 4.1, and 'three real-world domain' in the abstract should be 'three real-world domains'.
  2. [Section 4.1] The hand-crafted rules are only partially specified because the offensive words are redacted as '[racist word]', '[sexist word]', etc.; please include the actual terms or a complete rule set in an appendix so that the baseline is reproducible.
  3. [Section 4.1] The validation split is mentioned, but the paper does not state how it was used (e.g., for early stopping, hyperparameter selection, or threshold selection). Please clarify to rule out any selection based on the test set.
  4. [Section 3.3] The LTN implementation of similar/2 uses a 'shifted sigmoid' to model the similarity threshold, but the slope or temperature of the sigmoid and the way the threshold is encoded are not specified; adding these details would improve reproducibility.
  5. [Figures 3-5] The figures would benefit from more detailed captions explaining the SOM color scale and how the clusters around constants were identified, since the qualitative claims in Section 4.2 rely on these visualizations.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the F1 comparisons are empirical results from external pretrained embeddings and held-out test data; no derivation step reduces to its own inputs.

full rationale

The paper's central claim is an experimental comparison, not a formal derivation from an input that already contains the result. The method uses off-the-shelf external embeddings (GloVe trained on tweets, GenePT trained on text), defines similar/2 as cosine similarity with a user-specified threshold, learns TILDE decision trees, converts them to LTN rules, and fine-tunes the embeddings on the training set with standard supervised learning. No parameter is fitted to a subset and then renamed a prediction: the reported F1 scores are computed on held-out test splits after training, and the LTN refinement is applied to the training set only. There is no load-bearing self-citation: the cited TILDE and LTN references are external prior work by other authors, and no 'uniqueness theorem' or analogous claim is invoked to force the choice of method. The hand-selected similarity thresholds (0.75/0.7/0.5) and the undersampling choices are hyperparameter decisions; they raise questions about statistical soundness, especially since the results are single splits without significance tests, but they are not circular because the thresholds are not derived from or equivalent to the reported F1 outcomes. The paper's own data-provenance note acknowledges a possible but small contamination risk for public text embeddings; that is a limitation rather than a circular step. Overall, the empirical comparison stands on its own as an experimental claim, so no circularity is identified.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the choice of pretrained embeddings as the sole subsymbolic signal, on per-dataset thresholds tau and hyperparameters, and on the assumption that LTN fine-tuning improves rather than overfits the embeddings. No new theoretical entities are introduced; the similar/2 predicate is a method component, not an invented entity.

free parameters (5)
  • similar/2 cosine threshold tau = 0.75 (hate speech), 0.7 (spam), 0.5 (drug response)
    User-defined threshold that determines which pairs of constants are groundings of the similarity predicate; set per dataset in Section 4.1 with no sensitivity analysis.
  • TILDE minimal_cases (minimum leaf support) = 50 (hate speech), 50 (spam), 20 (drug response)
    Minimum number of cases per leaf, set per dataset (Section 4.1).
  • Adam learning rate = 1e-3 (hate speech), 1e-1 (spam), 5e-4 (drug response)
    Learning rate for LTN fine-tuning, set per dataset (Section 4.1).
  • Undersampling ratio = balanced training set (majority undersampled to minority count)
    Training data are undersampled to equal class counts for spam and drug response, changing the prior distribution (Section 4.1).
  • Gene expression binarization threshold = not specified
    Gene expressions are binarized 'via thresholding' with no threshold value or procedure given (Section 4.1).
assumptions (5)
  • domain assumption Cosine similarity over pretrained embedding vectors (GloVe/GenePT) is a meaningful proxy for label-relevant semantic similarity between constants.
    The similar/2 predicate is defined as cosine similarity >= tau (Section 3.2); the entire signal available to the symbolic learner beyond exact matches comes from this assumption.
  • domain assumption Pretrained embeddings have not already incorporated the target labels or data (data provenance).
    The paper argues GenePT is trained on text, not gene expression, and that leakage for text data should be small (Section 5); this is asserted, not verified.
  • standard math The TILDE tree to LTN rule conversion (Algorithms 1 and 2) correctly captures the tree semantics with fuzzy logic operators.
    The conversion is described algorithmically but not proven; the LTN optimization relies on it.
  • ad hoc to paper The inductive bias that membership predicates must be paired with similar/2 yields useful rules.
    Section 3.2 imposes this constraint on the learner; it is a design choice not derived from prior theory.
  • domain assumption LTN fine-tuning of embeddings on the training set does not overfit in a way that hurts test performance.
    The method updates embeddings to maximize satisfaction of rules on training data; generalization is assumed, with no regularization or early stopping described (Section 3.3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Symbolic Machine Learning by Subsymbolic Representations." pith.science (2026). https://pith.science/paper/QP22OJGG

@misc{pith2026250614569,
  author       = {Pith},
  title        = {Pith review of: Enhancing Symbolic Machine Learning by Subsymbolic Representations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QP22OJGG}},
  note         = {Machine review of arXiv:2506.14569}
}
read the original abstract

The goal of neuro-symbolic AI is to integrate symbolic and subsymbolic AI approaches, to overcome the limitations of either. Prominent systems include Logic Tensor Networks (LTN) or DeepProbLog, which offer neural predicates and end-to-end learning. The versatility of systems like LTNs and DeepProbLog, however, makes them less efficient in simpler settings, for instance, for discriminative machine learning, in particular in domains with many constants. Therefore, we follow a different approach: We propose to enhance symbolic machine learning schemes by giving them access to neural embeddings. In the present paper, we show this for TILDE and embeddings of constants used by TILDE in similarity predicates. The approach can be fine-tuned by further refining the embeddings depending on the symbolic theory. In experiments in three real-world domain, we show that this simple, yet effective, approach outperforms all other baseline methods in terms of the F1 score. The approach could be useful beyond this setting: Enhancing symbolic learners in this way could be extended to similarities between instances (effectively working like kernels within a logical language), for analogical reasoning, or for propositionalization.

Figures

Figures reproduced from arXiv: 2506.14569 by the authors.

Figure 1
Figure 1. Overview of the proposed system, integrating subsymbolic embeddings [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. LTN architecture for one rule component: The left-hand side reflects the [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Two TILDE decision trees harnessing the subsymbolic similar/2 predi￾cate. Left: spam classification Right: drug-response prediction. Since our optimization objective explicitly targets the F1 score, the stable accuracy results align with expectations, reflecting the prioritization of correctly classifying instances from less-represented classes [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Self organizing map visualization of sample words from the Spam dataset, [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Self organizing map visualization of sample words from the Spam dataset [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 6 canonical work pages

  1. [1]

    UCI Machine Learning Repository (2011)

    Almeida, T., Hidalgo, J.: SMS Spam Collection. UCI Machine Learning Repository (2011). https://doi.org/10.24432/C5CC84

    doi:10.24432/c5cc84 2011

  2. [2]

    Artificial Intelligence 303, 103649 (2022)

    Badreddine, S., d’Avila Garcez, A., Serafini, L., Spranger, M.: Logic tensor networks. Artificial Intelligence 303, 103649 (2022). https://doi.org/10.1016/j.artint.2021.103649

    arXiv 2022

  3. [3]

    Artificial intelligence 101(1-2), 285–297 (1998)

    Blockeel, H., De Raedt, L.: Top-down induction of first-order logical decision trees. Artificial intelligence 101(1-2), 285–297 (1998). https://doi.org/10.1016/S0004- 3702(98)00034-4

    doi:10.1016/s0004- 1998

  4. [4]

    bioRxiv pp

    Chen, Y., Zou, J.: Genept: a simple but effective foundation model for genes and cells built from chatgpt. bioRxiv pp. 2023–10 (2024). https://doi.org/10.1101/2023.10.16.562533

    doi:10.1101/2023.10.16.562533 2024

  5. [5]

    Evans, R., Grefenstette, E.: Learning explanatory rules from noisy data. J. Artif. Intell. Res. 61, 1–64 (2018). https://doi.org/10.1613/JAIR.5714

    doi:10.1613/jair.5714 2018

  6. [6]

    https://huggingface.co/fse/glove-twitter-200 (2023), accessed: 2025-05-28

    Fast Sentence Embeddings (fse): fse/glove-twitter-200 [word embedding model]. https://huggingface.co/fse/glove-twitter-200 (2023), accessed: 2025-05-28

    work page 2023

  7. [7]

    In: Besold, T.R., Kutz, O

    Frosst, N., Hinton, G.E.: Distilling a neural network into a soft decision tree. In: Besold, T.R., Kutz, O. (eds.) Proceedings of the First International Workshop on Comprehensibility and Explanation in AI and ML 2017 co-located with 16th (AI*IA 2017), , 2017. CEUR Workshop Proceedings, vol. 2071. CEUR-WS.org (2017). https://doi.org/10.48550/arXiv.1711.09...

    arXiv doi:10.48550/arxiv.1711.09784 2017

  8. [8]

    arXiv preprint (2023)

    Gibaut, W., Pereira, L., Grassiotto, F., Osorio, A., Gadioli, E., Munoz, A., Gomes, S., Santos, C.d.: Neurosymbolic ai and its taxonomy: a survey. arXiv preprint (2023). https://doi.org/10.48550/arXiv.2305.08876

    arXiv doi:10.48550/arxiv.2305.08876 2023

Show all 24 references
  1. [9]

    In: Proceed- ings of the 37th International Conference on Machine Learning, ICML 2020

    Hazimeh, H., Ponomareva, N., Mol, P., Tan, Z., Mazumder, R.: The tree ensemble layer: Differentiability meets conditional computation. In: Proceed- ings of the 37th International Conference on Machine Learning, ICML 2020. Proceedings of Machine Learning Research, vol. 119, pp....

    2020 doi arXiv

  2. [10]

    Cell166(3), 740–754 (2016)

    Iorio, F., Knijnenburg, T.A., Vis, D.J., Bignell, G.R., Menden, M.P., Schubert, M., Aben, N., Gonçalves, E., Barthorpe, S., Lightfoot, H., et al.: A landscape of pharmacogenomic interactions in cancer. Cell166(3), 740–754 (2016)

    2016

  3. [11]

    arXiv preprint arXiv:2009.10277 (2020)

    Kennedy, C.J., Bacon, G., Sahn, A., von Vacano, C.: Constructing in- terval variables via faceted rasch measurement and multitask deep learn- ing: a hate speech application. arXiv preprint arXiv:2009.10277 (2020). https://doi.org/10.48550/arXiv.2009.10277

    2020 doi arXiv

  4. [12]

    In: 2011 IEEE 11th international conference on data mining

    Khot, T., Natarajan, S., Kersting, K., Shavlik, J.: Learning markov logic networks via functional gradient boosting. In: 2011 IEEE 11th international conference on data mining. pp. 320–329. IEEE (2011). https://doi.org/10.1109/ICDM.2011.87

    2011 doi

  5. [13]

    In: Pro- ceedings of the 22nd international conference on Machine learning

    Kok, S., Domingos, P.: Learning the structure of markov logic networks. In: Pro- ceedings of the 22nd international conference on Machine learning. pp. 441–448 (2005). https://doi.org/10.1145/1102351.1102407

    2005

  6. [14]

    In: Kambhampati, S

    Kontschieder, P., Fiterau, M., Criminisi, A., Bulò, S.R.: Deep neural decision forests. In: Kambhampati, S. (ed.) Proceedings of the Twenty-Fifth Interna- tional Joint Conference on Artificial Intelligence, IJCAI 2016. pp. 4190–4194. IJ- CAI/AAAI Press (2016). https://doi.org/...

    2016 doi

  7. [15]

    Advances in neural information processing systems 31 (2018)

    Manhaeve, R., Dumancic, S., Kimmig, A., Demeester, T., De Raedt, L.: Deep- problog: Neural probabilistic logic programming. Advances in neural information processing systems 31 (2018). https://doi.org/10.48550/arXiv.1805.10872

    2018 doi arXiv

  8. [16]

    In: 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 (2019)

    Mao, J., Gan, C., Kohli, P., Tenenbaum, J.B., Wu, J.: The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision. In: 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 (2019). ht...

    2019 doi arXiv

  9. [17]

    Artificial Intelligence328, 104062 (2024)

    Marra, G., Dumančić, S., Manhaeve, R., De Raedt, L.: From statistical relational to neurosymbolic artificial intelligence: A survey. Artificial Intelligence328, 104062 (2024). https://doi.org/10.1016/j.artint.2023.104062

    2024

  10. [18]

    In: Proceedings of the 24th international conference on Machine learning

    Mihalkova, L., Mooney, R.J.: Bottom-up learning of markov logic network struc- ture. In: Proceedings of the 24th international conference on Machine learning. pp. 625–632 (2007). https://doi.org/10.1145/1273496.1273575

    2007

  11. [19]

    https://doi.org/10.48550/arXiv.1301.3781

    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word repre- sentations in vector space (2013). https://doi.org/10.48550/arXiv.1301.3781

    2013 doi arXiv

  12. [20]

    In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)

    Pennington, J., Socher, R., Manning, C.D.: Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). pp. 1532–1543 (2014). https://doi.org/10.3115/v1/D14-1162

    2014 doi

  13. [21]

    Machine learning62, 107– 136 (2006)

    Richardson, M., Domingos, P.: Markov logic networks. Machine learning62, 107– 136 (2006). https://doi.org/10.1007/s10994-006-5833-1

    2006 doi

  14. [22]

    Bioinformatics 35(14), i501–i509 (2019)

    Sharifi-Noghabi, H., Zolotareva, O., Collins, C.C., Ester, M.: Moli: multi-omics late integration with deep neural networks for drug response prediction. Bioinformatics 35(14), i501–i509 (2019). https://doi.org/10.1093/bioinformatics/btz318

    2019 doi

  15. [23]

    Roth et al

    Vettigli, G.: Minisom: minimalistic and numpy-based implementation of the self organizing map (2018), https://github.com/JustGlowing/minisom/ 14 S. Roth et al

    2018

  16. [24]

    Neural Networks 166, 105–126 (2023)

    Yu, D., Yang, B., Liu, D., Wang, H., Pan, S.: A survey on neural- symbolic learning systems. Neural Networks 166, 105–126 (2023). https://doi.org/10.1016/j.neunet.2023.06.028

    2023 doi

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.