Leveraging Interpretable Tsetlin Machine for PDF Malware Detection

Ole-Christoffer Granmo; Rahul Jaiswal

Leveraging Interpretable Tsetlin Machine for PDF Malware Detection

T0 review · 2 major / 0 minor · reviewed 2026-07-13 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A Tsetlin Machine detects PDF malware at 98 percent accuracy while explaining each decision through readable logical rules.

desk verdict Solid first application of Tsetlin Machines to PDF malware: competitive accuracy, real interpretability demos, and a new public dataset, tempered by single-corpus evaluation after heavy cleaning. read the letter →

arxiv 2607.09290 v2 pith:I6PYJDAE submitted 2026-07-10 cs.CR cs.LG

Rahul Jaiswal , Ole-Christoffer Granmo This is my paper

classification cs.CRcs.LG

keywords CybersecurityMalwareDetectionPortableDocumentFormatTsetlinMachineInterpretableLearningStaticAnalysis

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that PDF malware can be spotted by a Tsetlin Machine that learns human-readable logical clauses from static file features, without ever opening or running the document. On the RIT-PDFMal-2026 collection, after cleaning and balancing, the model reaches 98.02 percent accuracy and a macro F1 near 96 percent, staying competitive with Random Forest and gradient-boosting baselines while finishing inference in a few microseconds. Because every prediction is a transparent vote among propositional clauses, an analyst can see which rules and features drove the benign or malicious label. The work matters because PDFs remain a major attack channel and most high-performing detectors stay black boxes that need approximate post-hoc explanations. Intrinsic interpretability therefore turns competitive detection numbers into something security teams can audit and trust in practice.

What carries the argument

The Tsetlin Machine: a rule-based learner that builds conjunctive clauses from binary literals and their negations, then aggregates signed clause votes (bounded by threshold T) to decide the class. Those clauses are the human-readable decision rules that make classification transparent.

What would settle it

Re-run the identical feature extraction, preprocessing, and TM training pipeline on a fresh, independently collected PDF corpus that keeps natural class balance and includes malware families arriving after the original collection window; a large drop below the reported 98 percent accuracy would falsify the generalization claim.

Watch

Extended reading notes

Core claim

A Tsetlin Machine trained on 42 static numerical features extracted from PDF files without execution classifies them as benign or malicious at 98.02 percent accuracy (macro precision 96.03 percent, recall 95.95 percent, F1 95.99 percent) on the held-out RIT-PDFMal-2026 test set, matching or closely trailing strong black-box classifiers while supplying intrinsic explanations through clause activations, class-vote totals, and signed feature contributions.

Load-bearing premise

The 42 static numerical features, after discarding more than a third of samples as duplicates and undersampling the benign class, still capture the patterns that matter for real-world PDF malware.

Editorial extensions

If this is right

Security operators can inspect activated clauses and top feature contributions to audit why any given PDF was flagged or cleared.
Static analysis plus TM inference runs in roughly three microseconds per sample, supporting high-volume scanning.
Competitive accuracy is obtained without post-hoc explanation tools such as SHAP or LIME.
The same clause representation surfaces which structural PDF elements (JavaScript, Encrypt, Launch, XFA, etc.) dominate malicious patterns.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the model relies only on static structural counts, it can serve as a cheap first-stage filter before slower sandboxes or dynamic analysis.
The higher false-negative rate relative to false positives suggests some malware families remain under-covered by the learned clauses; expanding the clause budget or feature set is a direct next experiment.
Readable rules open a path for human-in-the-loop editing in which analysts disable or refine clauses that fire on known false positives.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a clean applied paper that puts Tsetlin Machines on PDF malware for the first time and ships the accompanying RIT-PDFMal-2026 corpus. That combination is the real novelty; neither the model nor static PDF features are new, but the concrete numbers, clause heatmaps, and public data are.

What it does well is straightforward. After standard preprocessing they get 98.02% accuracy and 95.99% macro F1, within a hair of Random Forest and ahead of XGBoost/LightGBM, with faster inference than RF. The interpretability section is not hand-waving: class-vote bars, clause-activation heatmaps, and signed feature contributions are shown for both correct and incorrect cases. That is more transparent than the usual SHAP/LIME post-hoc layer. The experimental pipeline is also transparent—stratified split, undersampling only on train, five-fold CV for hyper-parameters, class-wise metrics, confusion matrix.

The soft spots are real but ordinary. Everything rests on one dataset after discarding 36.84% duplicates and undersampling the majority class; the remaining malicious count is modest, and there is no ablation that restores the original balance or tests an external corpus. So the 98% figure and the learned clauses may be optimistic for live traffic. No code release, no significance tests, no user study of the explanations. Those are engineering caveats, not load-bearing contradictions.

Math and citations look fine: standard TM formulation, honest self-cites to the author’s radio-map work that do not prop up the malware claims, and a reasonable baseline set. Free parameters (clauses, T, s, bins) are reported.

This is useful for people who care about explainable static malware detectors and want a reproducible starting point. I would send it to peer review; a referee can demand the missing ablations and multi-corpus checks without the paper needing to be reinvented. Worth a look if you work in that niche; not required reading otherwise.

Referee Report

2 major / 0 minor

Summary. The paper proposes a Tsetlin Machine (TM) framework for static PDF malware detection. It extracts 42 numerical structural features without executing files, preprocesses the RIT-PDFMal-2026 corpus (duplicate removal, train-only undersampling, min-max scaling, KBinsDiscretizer binarization), and trains a TM with 250 clauses to classify benign vs. malicious PDFs. On a held-out 20% test split the model reports 98.02% accuracy and 95.99% macro F1, competitive with Random Forest (98.28%) and superior to XGBoost/LightGBM, while offering lower inference latency than RF and intrinsic interpretability via class-vote scores, clause-activation heatmaps, and feature-contribution plots. The authors position the combination of competitive accuracy, efficiency, and built-in explainability as the main contribution for practical PDF malware detection.

Significance. If the reported accuracy and interpretability claims hold under realistic distributions, the work supplies a concrete, low-latency alternative to black-box ensembles for a high-volume attack vector. The TM’s propositional clauses and vote/feature visualizations are a genuine methodological advantage over post-hoc SHAP/LIME explanations commonly applied to PDF detectors. The experimental pipeline is transparent (stratified split, train-only undersampling, five-fold CV, class-wise metrics, confusion matrix, seven baselines). The main limitation is that all evidence rests on a single, heavily cleaned public corpus; external validation or multi-dataset results would be required before the practical-deployment claim can be considered established. Within those bounds the contribution is solid and of clear interest to the malware-detection community.

major comments (2)

Section V-C1 and Tables II/IV: After discarding 8,966 duplicates (36.84% of the original 24,337 samples) only 2,222 malicious files remain. Random undersampling is then applied solely to the training split, so the absolute number of malicious test examples is small. No ablation restores the original class prior, re-inserts near-duplicates, or evaluates an external hold-out corpus. Consequently the headline 98.02% accuracy / 95.99% macro-F1 figures (and the learned clauses shown in Figs. 7–15) may be optimistic relative to live PDF streams; at minimum the paper should quantify sensitivity to these preprocessing choices or report results on an independent corpus.
Section V-D / Table VI: The state-of-the-art comparison juxtaposes methods evaluated on Contagio, Evasive-2022 and RIT-PDFMal-2026. Because the datasets differ in collection period, feature sets and class balance, the claim of “competitive performance … with existing methods” is only weakly supported. Either re-evaluate the baselines on the same RIT-PDFMal-2026 split or clearly qualify the comparison as non-head-to-head.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: purely empirical supervised classification with held-out test metrics; self-citations are unrelated and non-load-bearing.

full rationale

The paper's central claims (98.02% accuracy / 95.99% macro F1 on RIT-PDFMal-2026, competitive with RF, plus intrinsic TM interpretability via clauses/votes/feature contributions) are obtained by standard supervised training and evaluation: static feature extraction, duplicate removal, stratified train/test split, random undersampling only on train, min-max + KBinsDiscretizer binarization, TM training with fixed hyperparameters (clauses=250, T=15, s=5, 50 epochs), and measurement of accuracy/precision/recall/F1/confusion matrix/inference time on the untouched test set (Tables III–V, Figs. 5–6). There is no mathematical derivation, uniqueness theorem, or fitted constant that is later re-presented as a prediction. Self-citations ([7]–[9], [23], [24]) concern the author's prior radio-map transfer-learning and speech-quality work and do not underwrite any malware-detection equation, feature set, or performance number; the TM itself is cited to Granmo and the dataset to Alani. Preprocessing choices affect absolute numbers but do not create a by-construction identity between inputs and reported outputs. The evaluation is therefore self-contained against the stated benchmark.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The central empirical claim rests on standard ML modeling choices (binary classification, static features, undersampling) plus a handful of free hyper-parameters of the Tsetlin Machine and the binarizer. No new physical entities or unproved mathematical axioms are introduced; the load-bearing assumptions are domain conventions of static malware analysis and the representativeness of the single public dataset after aggressive cleaning.

free parameters (5)

number of clauses = 250
Set to 250; controls model capacity and is chosen empirically (Table III).
voting threshold T = 15
Set to 15; constrains accumulated clause votes (Eq. 2, Table III).
specificity s = 5
Set to 5; controls literal inclusion probability during training (Table III).
KBinsDiscretizer n_bins = 15
Set to 15 with quantile strategy; converts continuous features to binary literals required by TM (Table III).
training epochs = 50
Fixed at 50; convergence shown in Fig. 5 but not ablated.

assumptions (3)

domain assumption Static numerical features extracted without execution are sufficient to discriminate malicious from benign PDFs at the reported accuracy.
Invoked throughout Sections III–V; no dynamic analysis or sandbox execution is performed.
domain assumption Random undersampling of the majority class on the training split does not destroy the decision boundary needed for generalization to the original imbalanced test distribution.
Applied in Section V-C1; test set remains imbalanced while train is forced balanced.
domain assumption Propositional clauses learned by the Tsetlin Machine constitute faithful, human-interpretable explanations of the model’s decisions.
Core claim of intrinsic interpretability (Abstract, Section V-C3); no user study validates that analysts actually understand or trust the clauses.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Leveraging Interpretable Tsetlin Machine for PDF Malware Detection." pith.science (2026). https://pith.science/paper/I6PYJDAE

@misc{pith2026260709290,
  author       = {Pith},
  title        = {Pith review of: Leveraging Interpretable Tsetlin Machine for PDF Malware Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I6PYJDAE}},
  note         = {Machine review of arXiv:2607.09290}
}

read the original abstract

In the digital era, Portable Document Format (PDF) is one of the most widely used file formats for storing and exchanging digital documents due to its platform independence and rich functionality. However, these same capabilities have also made PDF files an attractive attack vector for cyberattackers, who embed malicious code within seemingly legitimate documents to compromise target systems. This paper presents a novel interpretable Tsetlin Machine (TM)-based framework for PDF malware detection. The proposed framework extracts salient features from PDF documents through static analysis without executing the files and employs rule-based learning to accurately classify benign and malicious PDF documents. Numerical evaluation on the RIT-PDFMal-2026 dataset demonstrates that the proposed framework achieves an accuracy of 98.02%, outperforming several state-of-the-art machine learning classifiers. Moreover, the proposed framework provides intrinsic interpretability by transparently explaining its classification decisions. Edge deployment on a Raspberry Pi further supports real-time, on-device PDF malware detection. The combination of better accuracy, computational efficiency, and intrinsic interpretability makes the proposed framework a promising solution for practical PDF malware detection.