REVIEW 4 major objections 4 minor 44 references
Cryptanalysis via Machine Learning Based Information Theoretic Metrics
T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read Neural networks using information-theoretic losses can act as a low-cost empirical auditor of encryption schemes, flagging non-IND-CPA ciphers and misconfigured CPA modes in a known-plaintext setting.
desk verdict The classifier experiments are a correct but unsurprising IND-CPA realization; the MI leakage estimates are unsupported because they contradict a basic information-theoretic identity. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Two load-bearing pieces carry the argument. The first is the Donsker–Varadhan representation of KL divergence, which expresses mutual information as a supremum over test functions and lets a neural network $F_\phi$ estimate a lower bound on $I(X;Y)$ from samples; the paper adds a stabilization term to control variance. The second is a binary cross-entropy classifier with a sigmoid output that maps a ciphertext to the probability that its plaintext was uniform rather than all-zero, trained by stochastic gradient descent. Both algorithms consume the same dataset structure: 100,000 known plaintext–ciphertext pairs per scheme, encrypted with a reused key, with 20,000 held out for testing. The small network has two hidden layers of 100 nodes and the large network four hidden layers of 600 nodes, and the comparison between them supports the paper's claim that small models suffice for vulnerability screening.
What would settle it
Train the same MI estimator on a deterministic cipher with a fixed key and uniform plaintext, where the true mutual information equals the plaintext entropy (about 11.1 nats in their setup). If the estimate plateaus near the small values in Table II instead of approaching the true value, the reported leakage figures are model-capacity artifacts rather than cryptosystem properties.
Extended reading notes
Core claim
The paper's central claim, stated on its own terms, is that a neural-network binary classifier trained with binary cross entropy on labeled ciphertexts can act as a concrete adversary in an IND-CPA indistinguishability game. The classifier is trained on ciphertexts produced from two plaintext classes—uniform random bits and all-zero bits—and then asked to guess which class a challenge ciphertext came from. Reported test accuracies are 100% for deterministic DES, AES-ECB, and plain RSA; about 50% for non-deterministic DES, AES-CTR, and padded RSA; and up to 99.5% for deliberately faulty variants such as a reduced counter field or reused OAEP padding. The companion MI estimator, built on the regularized Donsker–Varadhan objective, reports higher leakage for deterministic than randomized modes, which the paper reads as evidence that the estimators capture real dependencies in the known-plaintext setting.
Load-bearing premise
The load-bearing premise is that the neural estimates of mutual information and the classifier accuracies reflect properties of the cryptosystem rather than the network's capacity, initialization, or training dynamics; the paper's own observation that larger networks report larger MI values is the point where that premise is most exposed.
Editorial extensions
If this is right
- An auditor can use a cheap small-network classifier as triage: only when it flags a scheme is it worth deploying larger, more expensive models that might fully break the cipher.
- The same framework catches implementation faults—counter resets, padding reuse—that undermine otherwise CPA-secure ciphers, making it usable for continuous monitoring of deployed systems.
- Because the algorithms need only known plaintext–ciphertext pairs and a choice of two plaintext classes, they apply to any cryptosystem without scheme-specific cryptanalysis, including network-coding-based designs.
- The MI estimates provide a relative ordering of leakage across schemes and modes, so security comparisons can be made on a common empirical scale.
Reading between the lines
- The reported MI values are best read as relative lower bounds rather than absolute leakage: for a deterministic cipher with a fixed key and uniform plaintext the true mutual information equals the plaintext entropy, far above the values in Table II, so the practical signal is the ordering across schemes.
- A stronger indistinguishability test would use two random plaintext classes instead of all-zero versus uniform; the accuracy drop under that variant would show how much of the classifier's success comes from exploiting the specific all-zero distribution.
- The same BCE machinery could be pointed at IND-CCA-style games or at membership-inference audits, since it only requires labeled ciphertext samples from two groups.
- The 100% classification scores should be understood as indistinguishability breaks, not key recovery or plaintext recovery; the paper's 'sanity check' framing is the right scope.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes two machine-learning tools for empirical cryptosystem auditing in a known/chosen-plaintext setting: MINE-based estimation of mutual information between plaintext and ciphertext, and a binary cross-entropy classifier intended to model an IND-CPA indistinguishability game. Experiments are run on baselines (no encryption, one-time pad, constant-XOR), DES, AES-ECB/CTR, plain/padded RSA, faulty variants (counter reset, padding reuse), and the HUNCC network-coding cryptosystem. The authors claim that the classifier correctly flags non-IND-CPA schemes and that smaller networks can identify vulnerabilities with less computing power, providing an empirical security bound.
Significance. A reliable ML-based empirical audit would be valuable, and the classification experiment is a plausible sanity check: it uses held-out test sets, reports near-chance accuracy for CPA-secure modes, and reports high accuracy for deterministic modes and for schemes with faulty parameters. However, the MI-leakage half of the paper is not supportable: the reported estimates contradict elementary information-theoretic facts under the paper's own experimental setup, and the paper itself attributes differences in estimates to network capacity. Because the MI-based 'empirical bound' is a central claimed contribution, this is a load-bearing flaw. The BCE classifier results, while internally plausible, are not sufficient to rescue the paper's overall claims as stated.
major comments (4)
- [Section III-A, Section IV-A, Tables I and II] Under the setup of Section III-A, the same key is reused for all encryptions. For any tested scheme in which the ciphertext uniquely determines the plaintext (DES, AES-ECB, plain RSA, and even constant-XOR), the true mutual information satisfies I(X;Y)=H(X); the paper itself gives H(X)=11.1 nats for its no-encryption baseline. Table I reports only 5.16 nats for constant-XOR, and Table II reports 0.733, 0.0621, and 0.7285 nats for DES, AES-ECB, and plain RSA. These values are one to two orders of magnitude below the true value, so the estimates cannot be interpreted as leakage measures. The no-encryption baseline estimate of 9.17 nats also shows that the estimator is not calibrated. This invalidates the MI-based empirical-bound claims in the abstract and Section IV-B.
- [Section IV-B] The paper explains that the larger network 'is able to calculate a larger MI leakage,' with AES-ECB moving from 0.0621 to 0.7068 nats. Mutual information leakage of a cryptosystem is a fixed quantity; if the estimate moves by more than a factor of ten with network size, the reported metric is a property of the model and training procedure, not of the cryptosystem. This is direct evidence that the MI results track model capacity, and it further undermines any interpretation of the numbers as security bounds.
- [Section III-A and Table I] The one-time-pad baseline contradicts the stated fixed-key protocol. With a single reused key, 'one-time pad' is identical to constant-XOR and would give I(X;Y)=H(X)=11.1 nats and 100% classification accuracy; the reported values of 0.0092 nats and 50.36% are only possible with a fresh key per message. The paper needs to state the key schedule explicitly, because the baseline comparison is otherwise incoherent.
- [Section III-D and Section V] The BCE classifier is not an implementation of the IND-CPA game. It is a supervised classifier trained on a fixed labeled dataset and does not perform the adaptive challenge phase of Definition II-A2. Therefore, high accuracy on deterministic schemes is a valid demonstration that those schemes admit a distinguisher for the two chosen plaintext distributions, but a near-50% result for AES-CTR or padded RSA does not 'confirm' IND-CPA security (Section V): it only shows that this classifier fails on this pair of distributions. The paper should soften these security claims.
minor comments (4)
- [Section III-B heading] The heading 'CRYTPO-MINE' contains a typo; it should read 'CRYPTO-MINE.'
- [Section II-B2] The BCE definition swaps the roles of the true and predicted distributions: the standard loss is BCE(p, q) = -[p log q + (1-p) log(1-q)] with p the true label and q the predicted probability, whereas the paper writes BCE(ˆp, p) = -[ˆp log p + (1-ˆp) log(1-p)]. This makes Algorithm 2's notation confusing and should be corrected.
- [Figures 2-4] The epoch axes in Figures 2-4 are garbled in the arXiv rendering (for example, '120140160180110011201140116011801'), making it impossible to read the training curves. The figures should be regenerated with legible tick labels.
- [Section IV-C] The sentence 'only classifying 204 out of the 20,000 ciphers' is clearer as an accuracy value (approximately 98.98%), consistent with the percentages reported in Table III.
Circularity Check
No significant circularity found; the empirical pipelines use held-out evaluation and the self-cited theorems are independent published results.
full rationale
The paper's derivation chain is not circular. The MI-leakage estimates are produced by a standard neural estimator (MINE with a stabilization term) trained on plaintext-ciphertext pairs and evaluated on a separate 20,000-sample test set; the reported values are not fitted parameters renamed as predictions. The BCE classifier is likewise trained on labeled ciphertexts and its accuracy is measured on held-out samples, so the 'predictions' of non-IND-CPA status are generalizations, not identities with the training objective. The paper's reliance on the authors' prior Crypto-mine framework and on HUNCC security theorems ([35], [38]) is interpretive: those theorems are published, independently stated results used to explain why the HUNCC experiments behave as they do, and the empirical outcomes do not reduce to the citations. The deterministic-scheme 100% accuracies are explicitly acknowledged as expected known facts, and the large discrepancy between the reported MI estimates and the paper's own H(X)=11.1 nats upper bound for deterministic schemes is a correctness/sanity-check problem rather than a circularity-of-derivation problem. No equation in the paper defines a predicted quantity in terms of the same quantity, and no fitted parameter is relabeled as a prediction.
Assumptions & free parameters
free parameters (5)
- MINE stabilization coefficient =
0.1
- Small and large network architectures =
2x100 nodes; 4x600 nodes
- AES-CTR counter reset period =
100,000 samples
- OAEP padding reuse period =
100,000 samples
- Training hyperparameters =
lr=1e-4, 1000 epochs, batch 10,000
assumptions (6)
- standard math Donsker-Varadhan representation of KL divergence
- domain assumption MINE objective with the 0.1 stabilization term converges to a useful lower bound
- domain assumption Zero MI implies perfect secrecy and MI leakage indicates meaningful insecurity
- domain assumption The fixed-key, two-class supervised dataset faithfully models an IND-CPA game
- domain assumption HUNCC security theorems from [35] and [38] are valid
- domain assumption The implemented AES, DES, RSA, and OAEP are correct except for the introduced faults
Cite this review
Pith. "Pith review of Cryptanalysis via Machine Learning Based Information Theoretic Metrics." pith.science (2026). https://pith.science/paper/U7TI2R3X
@misc{pith2026250115076,
author = {Pith},
title = {Pith review of: Cryptanalysis via Machine Learning Based Information Theoretic Metrics},
year = {2026},
howpublished = {\url{https://pith.science/paper/U7TI2R3X}},
note = {Machine review of arXiv:2501.15076}
}
read the original abstract
The fields of machine learning (ML) and cryptanalysis share an interestingly common objective of creating a function, based on a given set of inputs and outputs. However, the approaches and methods in doing so vary vastly between the two fields. In this paper, we explore integrating the knowledge from the ML domain to provide empirical evaluations of cryptosystems. Particularly, we utilize information theoretic metrics to perform ML-based distribution estimation. We propose two novel applications of ML algorithms that can be applied in a known plaintext setting to perform cryptanalysis on any cryptosystem. We use mutual information neural estimation to calculate a cryptosystem's mutual information leakage, and a binary cross entropy classification to model an indistinguishability under chosen plaintext attack (CPA). These algorithms can be readily applied in an audit setting to evaluate the robustness of a cryptosystem and the results can provide a useful empirical bound. We evaluate the efficacy of our methodologies by empirically analyzing several encryption schemes. Furthermore, we extend the analysis to novel network coding-based cryptosystems and provide other use cases for our algorithms. We show that our classification model correctly identifies the encryption schemes that are not IND-CPA secure, such as DES, RSA, and AES ECB, with high accuracy. It also identifies the faults in CPA-secure cryptosystems with faulty parameters, such a reduced counter version of AES-CTR. We also conclude that with our algorithms, in most cases a smaller-sized neural network using less computing power can identify vulnerabilities in cryptosystems, providing a quick check of the sanity of the cryptosystem and help to decide whether to spend more resources to deploy larger networks that are able to break the cryptosystem.
Figures
Reference graph
Works this paper leans on
-
[1]
Crypto-mine: Cryptanalysis via mu- tual information neural estimation,
B. D. Kim, V . A. Vasudevan, J. Woo, A. Cohen, R. G. D’Oliveira, T. Stahlbuhk, and M. M ´edard, “Crypto-mine: Cryptanalysis via mu- tual information neural estimation,” in ICASSP 2024-2024 IEEE In- ternational Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 4820–4824
work page 2024
-
[2]
The im- pact of quantum computing on present cryptography,
V . Mavroeidis, K. Vishi, M. D. Zych, and A. Jøsang, “The im- pact of quantum computing on present cryptography,” arXiv preprint arXiv:1804.00200, 2018
arXiv 2018
-
[3]
L. Chen, L. Chen, S. Jordan, Y .-K. Liu, D. Moody, R. Peralta, R. A. Perlner, and D. Smith-Tone, Report on post-quantum cryptography . US Department of Commerce, National Institute of Standards and Technology . . . , 2016, vol. 12
work page 2016
-
[4]
Crystals-kyber: a cca-secure module-lattice-based kem,
J. Bos, L. Ducas, E. Kiltz, T. Lepoint, V . Lyubashevsky, J. M. Schanck, P. Schwabe, G. Seiler, and D. Stehl ´e, “Crystals-kyber: a cca-secure module-lattice-based kem,” in 2018 IEEE European Symposium on Security and Privacy (EuroS&P) . IEEE, 2018, pp. 353–367
2018
-
[5]
T. Prest, P.-A. Fouque, J. Hoffstein, P. Kirchner, V . Lyubashevsky, T. Pornin, T. Ricosset, G. Seiler, W. Whyte, and Z. Zhang, “Falcon,” Post-Quantum Cryptography Project of NIST , 2020
work page 2020
-
[6]
Breaking a fifth-order masked implementation of crystals-kyber by copy-paste,
E. Dubrova, K. Ngo, J. G ¨artner, and R. Wang, “Breaking a fifth-order masked implementation of crystals-kyber by copy-paste,” inProceedings of the 10th ACM Asia Public-Key Cryptography Workshop , 2023, pp. 10–20
work page 2023
-
[7]
Salsa: Attacking lattice cryptography with transformers,
E. Wenger, M. Chen, F. Charton, and K. E. Lauter, “Salsa: Attacking lattice cryptography with transformers,” Advances in Neural Information Processing Systems, vol. 35, pp. 34 981–34 994, 2022. 12
work page 2022
-
[8]
D. R. Stinson and M. Paterson, Cryptography: Theory and Practice , 4th ed. Chapman and Hall/CRC Press, 2018
work page 2018
Show all 44 references
-
[9]
Improving attacks on round-reduced speck32/64 using deep learning,
A. Gohr, “Improving attacks on round-reduced speck32/64 using deep learning,” in Advances in Cryptology–CRYPTO 2019: 39th Annual International Cryptology Conference, Santa Barbara, CA, USA, August 18–22, 2019, Proceedings, Part II 39 . Springer, 2019, pp. 150–179
2019
-
[10]
Differential privacy,
C. Dwork, “Differential privacy,” in International colloquium on au- tomata, languages, and programming . Springer, 2006, pp. 1–12
2006
-
[11]
Differential privacy: A survey of results,
——, “Differential privacy: A survey of results,” in International con- ference on theory and applications of models of computation . Springer, 2008, pp. 1–19
2008
-
[12]
Privacy loss in apple’s implementation of differential privacy on macos 10.12,
J. Tang, A. Korolova, X. Bai, X. Wang, and X. Wang, “Privacy loss in apple’s implementation of differential privacy on macos 10.12,” arXiv preprint arXiv:1709.02753, 2017
2017 arXiv
-
[13]
Differential privacy at scale: Uber and berkeley collaboration,
J. Near, “Differential privacy at scale: Uber and berkeley collaboration,” in Enigma 2018 (Enigma 2018) , 2018
2018
-
[14]
The us census bureau adopts differential privacy,
J. M. Abowd, “The us census bureau adopts differential privacy,” in Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining , 2018, pp. 2867–2867
2018
-
[15]
Tight auditing of differentially private machine learning,
M. Nasr, J. Hayes, T. Steinke, B. Balle, F. Tram `er, M. Jagielski, N. Car- lini, and A. Terzis, “Tight auditing of differentially private machine learning,” in 32nd USENIX Security Symposium (USENIX Security 23) , 2023, pp. 1631–1648
2023
-
[16]
Privacy auditing with one (1) training run,
T. Steinke, M. Nasr, and M. Jagielski, “Privacy auditing with one (1) training run,” Advances in Neural Information Processing Systems , vol. 36, 2024
2024
-
[17]
Shimeall and J
T. Shimeall and J. Spring, Introduction to information security: a strategic-based approach. Newnes, 2013
2013
-
[18]
Mine: mutual information neural estimation,
M. I. Belghazi, A. Baratin, S. Rajeswar, S. Ozair, Y . Bengio, A. Courville, and R. D. Hjelm, “Mine: mutual information neural estimation,” arXiv preprint arXiv:1801.04062 , 2018
2018 arXiv
-
[19]
Privacy-preserving aggregate mobility data release: An information-theoretic deep reinforcement learning approach,
W. Zhang, B. Jiang, M. Li, and X. Lin, “Privacy-preserving aggregate mobility data release: An information-theoretic deep reinforcement learning approach,” IEEE Transactions on Information Forensics and Security, vol. 17, pp. 849–864, 2022
2022
-
[20]
Bottle- necks club: Unifying information-theoretic trade-offs among complexity, leakage, and utility,
B. Razeghi, F. P. Calmon, D. Gunduz, and S. V oloshynovskiy, “Bottle- necks club: Unifying information-theoretic trade-offs among complexity, leakage, and utility,” IEEE Transactions on Information Forensics and Security, vol. 18, pp. 2060–2075, 2023
2023
-
[21]
A variational approach to privacy and fairness,
B. Rodr ´ıguez-G´alvez, R. Thobaben, and M. Skoglund, “A variational approach to privacy and fairness,” in 2021 IEEE Information Theory Workshop (ITW). IEEE, 2021, pp. 1–6
2021
-
[22]
Infoshape: Task-based neural data shaping via mutual information,
H. Esfahanizadeh, W. Wu, M. Ghobadi, R. Barzilay, and M. M ´edard, “Infoshape: Task-based neural data shaping via mutual information,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2023, pp. 1–5
2023
-
[23]
Texshape: Information theoretic sentence embedding for language models,
H. K. Kale, H. Esfahanizadeh, N. Elias, O. Baser, M. Medard, and S. Vishwanath, “Texshape: Information theoretic sentence embedding for language models,” arXiv preprint arXiv:2402.05132 , 2024
2024 arXiv
-
[24]
A theoretical treatment of related-key attacks: Rka-prps, rka-prfs, and applications,
M. Bellare and T. Kohno, “A theoretical treatment of related-key attacks: Rka-prps, rka-prfs, and applications,” in International Conference on the Theory and Applications of Cryptographic Techniques. Springer, 2003, pp. 491–506
2003
-
[25]
Introduction to side-channel attacks,
F.-X. Standaert, “Introduction to side-channel attacks,” Secure integrated circuits and systems , pp. 27–42, 2010
2010
-
[26]
Power side-channel attack analysis: A review of 20 years of study for the layman,
M. Randolph and W. Diehl, “Power side-channel attack analysis: A review of 20 years of study for the layman,” Cryptography, vol. 4, no. 2, p. 15, 2020
2020
-
[27]
Liang, H
Y . Liang, H. V . Poor, and S. Shamai, Information theoretic security . Now Publishers Inc, 2009
2009
-
[28]
Communication theory of secrecy systems,
C. E. Shannon, “Communication theory of secrecy systems,” The Bell system technical journal , vol. 28, no. 4, pp. 656–715, 1949
1949
-
[29]
Goodfellow, Y
I. Goodfellow, Y . Bengio, and A. Courville, Deep learning. MIT press, 2016
2016
-
[30]
Regularized mutual information neural estimation,
K. Choi and S. Lee, “Regularized mutual information neural estimation,” 2020
2020
-
[31]
Entropy measures and unconditional security in cryptogra- phy,
C. Cachin, “Entropy measures and unconditional security in cryptogra- phy,” Ph.D. dissertation, ETH Zurich, 1997
1997
-
[32]
Data encryption standard,
D. E. Standard et al., “Data encryption standard,” Federal Information Processing Standards Publication, vol. 112, p. 3, 1999
1999
-
[33]
Aes proposal: Rijndael,
J. Daemen and V . Rijmen, “Aes proposal: Rijndael,” 1999
1999
-
[34]
A method for obtaining digital signatures and public-key cryptosystems,
R. L. Rivest, A. Shamir, and L. Adleman, “A method for obtaining digital signatures and public-key cryptosystems,” Communications of the ACM, vol. 21, no. 2, pp. 120–126, 1978
1978
-
[35]
Network coding-based post-quantum cryptography,
A. Cohen, R. G. L. D’Oliveira, S. Salamatian, and M. M ´edard, “Network coding-based post-quantum cryptography,” IEEE journal on selected areas in information theory , vol. 2, no. 1, pp. 49–64, 2021
2021
-
[36]
A mathematical theory of cryptography,
C. E. Shannon, “A mathematical theory of cryptography,” Bell System Technical Memo MM 45-110-02 , 1945
1945
-
[37]
Optimal asymmetric encryption,
M. Bellare and P. Rogaway, “Optimal asymmetric encryption,” in Advances in Cryptology—EUROCRYPT’94: Workshop on the Theory and Application of Cryptographic Techniques Perugia, Italy, May 9–12, 1994 Proceedings 13 . Springer, 1995, pp. 92–111
1994
-
[38]
Partial encryption after encoding for security and reliability in data systems,
A. Cohen, R. G. L. D’Oliveira, K. R. Duffy, and M. M ´edard, “Partial encryption after encoding for security and reliability in data systems,” in 2022 IEEE International Symposium on Information Theory (ISIT) . IEEE, 2022, pp. 1779–1784
2022
-
[39]
CERMET: Coding for Energy Reduction with Multiple Encryption Techniques– It’s easy being green,
J. Woo, V . A. Vasudevan, B. Kim, A. Cohen, R. G. L. D’Oliveira, T. Stahlbuhk, and M. M´edard, “CERMET: Coding for Energy Reduction with Multiple Encryption Techniques– It’s easy being green,” arXiv preprint arXiv:2308.05063, 2023
2023 arXiv
-
[40]
Coding-based hybrid post-quantum cryptosystem for non-uniform information,
S. Tarnopolsky and A. Cohen, “Coding-based hybrid post-quantum cryptosystem for non-uniform information,” in 2024 IEEE International Symposium on Information Theory (ISIT) , 2024, pp. 1830–1835
2024
-
[41]
Learning deep representations by mutual information estimation and maximization,
R. D. Hjelm, A. Fedorov, S. Lavoie-Marchildon, K. Grewal, P. Bach- man, A. Trischler, and Y . Bengio, “Learning deep representations by mutual information estimation and maximization,” arXiv preprint arXiv:1808.06670, 2018
2018 arXiv
-
[42]
Decipherment of substitution ciphers with neural lan- guage models,
N. Kambhatla, “Decipherment of substitution ciphers with neural lan- guage models,” 2018
2018
-
[43]
Post quantum cryptography,
NIST, “Post quantum cryptography,” NIST Computer Security Resource Center, Tech. Rep., 2017. [Online]. Available: https://csrc.nist.gov/projects/post-quantum-cryptography/ post-quantum-cryptography-standardization/evaluation-criteria/ security-(evaluation-criteria)
2017
-
[44]
Secure multi-source multicast,
A. Cohen, A. Cohen, M. Medard, and O. Gurewitz, “Secure multi-source multicast,” IEEE Transactions on Communications , vol. 67, no. 1, pp. 708–723, 2018
2018
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.