REVIEW 3 major objections 5 minor 37 references
Machine-learning octet $AB$-type binary compounds across chemical space with domain knowledge of the interatomic bond
T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read The paper claims that adding bond-aware recursion coefficients to atomic features more than halves the machine-learning error for predicting rocksalt-versus-zincblende stability of octet AB compounds.
desk verdict Bond-informed recursion features are a credible advance for RS/ZB stability prediction, but the headline RMSE numbers are inflated by pre-split feature selection and should not be taken at face value. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the recursion representation of the local electronic density of states: a Lanczos algorithm transforms the tight-binding Hamiltonian into a tridiagonal matrix whose diagonal and off-diagonal entries $a_n$ and $b_n$ encode increasingly distant coordination shells as a continued fraction. The chemistry enters through pairwise Hamiltonians taken from downfolded DFT eigenstates of diatomic molecules, so each A-B pair carries bond-specific domain knowledge. The paper converts these coefficients into per-compound features by aggregating site-resolved values (max, min, average) and taking differences and ratios between rocksalt and zincblende, producing the feature ladder AtomF+RecN. The argument is carried by an ensemble protocol: 20 independently trained KRR or SISSO (sure-independence-screening-and-sparsifying-operator) models per split, combined by inverse-MSE weighting, evaluated on 150 repeated 90/10 splits, with sequential feature selection (SFS) reducing each feature set before training.
What would settle it
Repeat the same 150-split ensemble protocol with sequential feature selection performed separately inside every training fold; if the AtomF+Rec6 test RMSE on the 78-compound dataset rises substantially above 0.029 eV, the claimed improvement over the reference is partly an artifact of selection leakage.
Extended reading notes
Core claim
The paper's central claim is that explicitly incorporating interatomic-bond information into the feature vector systematically improves prediction of the formation-energy difference $\Delta E_{AB}=E_{\mathrm{RS}}-E_{\mathrm{ZB}}$ between rocksalt and zincblende/wurtzite structures for the 82 octet $AB$ compounds used in Ref. [3]. The bond information is carried by recursion coefficients $a_n$ and $b_n$, obtained by Lanczos tridiagonalization of a tight-binding bond model with chemistry-specific Hamiltonians transferred from DFT eigenstates of diatomic molecules. Features are built by comparing site-resolved recursion coefficients between the two structures, via max/min/average aggregation and RS-ZB differences and ratios, concatenated with atomic features (AtomF) in increasing recursion order (Rec1 through Rec8). Using a 150-split ensemble of kernel-ridge regression (KRR) and SISSO (a compressed-sensing symbolic-regression method) models, the paper shows the bond-aware KRR ensemble beats both the previous three-feature symbolic-regression model and symbolic descriptors built from the same primary features. The best results are AtomF+Rec6 on the 78-compound refined dataset (test RMSE 0.029 eV) and AtomF+Rec4 on the full dataset (test RMSE 0.046 eV).
Load-bearing premise
The load-bearing premise is that the reported cross-validated errors are honest, even though the features are selected once on the full dataset before the train/test splits are generated, so the held-out samples have already influenced which features were chosen.
Editorial extensions
If this is right
- On the refined 78-compound dataset, adding recursion features up to order 6 lowers the mean test RMSE from about 0.080 eV for the reference descriptor to 0.029 eV, and the worst-case test MAE from 0.155 eV to 0.049 eV.
- On the full 82-compound dataset, the best bond-aware model reaches a mean test RMSE of 0.046 eV against 0.139 eV for the reference model re-evaluated in this paper's ensemble, so the improvement does not depend on removing the four outliers.
- The 3D SISSO ensemble built from the same features also improves on the reference but levels off around 0.062-0.074 eV test RMSE on the refined dataset, indicating that the gain is primarily due to the recursion features themselves rather than to the symbolic-regression wrapper.
- Permutation feature importance places lower-order recursion features such as $(a_3)^{\max}_r$ alongside ionization potentials and HOMO energies, suggesting the model is exploiting bond-strength differences between the two structures.
- Recursion orders 7 and 8 add little and can slightly raise mean RMSE, so the first six recursion levels capture essentially all the bonding information useful for these simple binaries.
Reading between the lines
- Because the recursion features do not require self-consistent DFT on the target structure, the same recipe could plausibly transfer to other structural competitions, such as wurtzite versus zincblende or layered versus cubic polymorphs, reusing the same per-element Hamiltonians.
- The sharp plateau beyond Rec6 suggests a practical rule of thumb for small binary datasets: six recursion levels suffice, which could be tested as a design guideline for larger feature-engineering studies.
- The quantitative margin may be partly optimistic because sequential feature selection was run once on the full dataset before the splits; a strictly nested feature-selection rerun would reveal whether the reported RMSEs hold or need revision.
- A strong transfer test would be to apply the best AtomF+Rec6 model to newly computed octet binaries outside the original 82, such as pairs involving heavier p-block or post-transition elements, and check whether the sub-0.03 eV accuracy persists.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a feature representation for octet AB binary compounds that combines elemental atomic features with recursion coefficients of a tight-binding bond model, where the pairwise Hamiltonians are downfolded from DFT dimer eigenstates. The authors evaluate kernel-ridge and SISSO ensemble models on the 82-compound dataset of Ghiringhelli et al. and on an outlier-reduced 78-compound version, using 150 repeated 90/10 splits and sequential feature selection. They report that adding recursion features up to order 4 (dataset D1) or order 6 (dataset D2) lowers the mean test RMSE from 0.139 eV to 0.046 eV and from 0.080 eV to 0.029 eV, respectively, relative to a reference model based on the three features of Ghiringhelli et al.
Significance. The core idea is attractive: recursion coefficients provide a physically motivated, target-independent description of the local bonding environment, and the paper asks a genuinely useful question about whether bond-level domain knowledge helps small-data stability prediction. The consistent monotonic improvement with recursion order on both datasets, the use of the same DFT data as Ref. 3, and the detailed per-compound analysis of failing splits are strengths. If the reported errors survive a nested feature-selection and outlier-removal protocol, the result would be a practically valuable demonstration of chemistry-aware descriptors for materials informatics. At present, however, the quantitative outperformance claim rests on a validation protocol that leaks test information into model construction, so the manuscript needs revision before the central quantitative claim can be accepted.
major comments (3)
- [II E and Fig. 4] Section II E states that 'The SFS is performed once on the complete dataset, prior to the train-test splits described below.' Because SFS scores candidate feature subsets by 10-fold KRR MSE computed on all 82/78 compounds, the held-out test targets participate in choosing the features. The reported test RMSE is therefore not an unbiased estimate of generalization, and the comparison is asymmetric: the AtomF+RecN models use data-driven SFS, while the Ghiringhelli et al. reference uses fixed features, so part of the reported improvement from 0.080 eV to 0.029 eV could be an artifact of selection leakage. Please move SFS inside each outer training split or otherwise demonstrate that the selected features are insensitive to held-out targets, and report the re-estimated RMSE.
- [II D and II E] The outlier-removal procedure in Section II D uses the target variable in its first stage and the full descriptor set including recursion features in its second stage, both before any train/test split; the test folds of dataset D2 are therefore defined on a population already cleaned with test information. In addition, Section II E says that splits with negative R2 'were ruled out when averaging the metrics reported below,' which biases the reported means if the reference and the proposed models fail on different fractions of splits. Please report aggregate metrics over all 150 splits and use a nested or training-only outlier-removal rule, or quantitatively show that these choices do not affect the conclusion.
- [III A-B, Figs. 5-6] The selection of Rec4 on D1 and Rec6 on D2 as the best recursion order is made by inspecting test-set performance on the very same 150 splits that define the reported errors. This is model selection on the test set and can bias the reported 'best' RMSE downward. Also, the reference model's D1 test RMSE of 0.139±0.088 eV differs substantially from the 0.08 eV originally reported for the same three-feature descriptor in Ref. 3, and the manuscript does not explain whether the reference was re-evaluated under the same split protocol or what causes this discrepancy; please clarify this in the revision.
minor comments (5)
- [Figs. 5-6] The x-axis labels in Figures 5 and 6 are truncated, with entries such as 'AtomF+Rec' missing the recursion order, which makes it difficult to verify the claimed monotonic trend; please provide complete labels or a legend.
- [Fig. 4] The flowchart places the SFS step outside the outer split loop, which visually encodes the leakage problem identified above; the figure should be updated to reflect the corrected validation protocol.
- [Eq. (2)] Equation (2) uses inconsistent notation for the averaged recursion coefficient (as_n + 3ap_n), mixing subscripts and superscripts; please harmonize the notation throughout.
- [Supplementary Figures S1-S6] The orbital-resolved (s/p) KRR curves in the supplementary figures are shown without error bars or summary statistics, unlike the other curves; please report the underlying values in a table.
- [Data availability] The data availability statement only promises that data and notebooks will be made available to reviewers and public upon acceptance; for reproducibility, please provide a permanent repository link or an anonymized copy in the revision.
Circularity Check
Full-data sequential feature selection and target-based outlier removal make the reported CV errors optimistically biased, but the recursion features are not fitted to the target; no definitional circularity.
-
fitted input called prediction
[Section II E, Machine-learning architecture]
"Candidate subsets are scored by the mean squared error of a KRR model with a polynomial kernel in a 10-fold cross-validation, and the subset size with the best score is retained. ... The SFS is performed once on the complete dataset, prior to the train-test splits described below, and the selected features are then kept fixed for all subsequent splits."
The feature subsets for AtomF+RecN are chosen using KRR MSE on the complete dataset, including the samples that later form each 10% test fold. Consequently, the reported held-out test RMSEs (0.046 eV on D1 and 0.029 eV on D2) are not unbiased estimates for a pipeline blind to those test labels; the feature space has been fitted to the test targets before the splits are defined. This biases the comparison against the fixed-feature Ghiringhelli reference in favor of AtomF+RecN. The final KRR weights are retrained per split and the recursion features are independent of the 82 formation energies, so the models are not literally predicting fitted values; a nested SFS is needed to confirm the direction and size of the improvement.
-
other
[Section II D, Data pre-processing]
"First, we used only the target variable together with the AtomF descriptors. BN and C2 appear as outliers due to exceptionally large positive Delta E ... By identifying these four outliers, we constructed two versions of the dataset: dataset 1 (D1) with all 82 data points and dataset 2 (D2) with 78 data points after removing all outliers."
The refined dataset D2, on which the headline 0.080 to 0.029 eV reduction is reported, is constructed by removing compounds flagged as outliers using the target variable before any train-test split. The AtomF+Rec6 model is therefore never tested on the four hardest-to-predict compounds, and the D2 test error is conditional on the full target distribution. This inflates the apparent improvement and means the quantitative claim is not a held-out assessment for the original 82-compound problem; it does not, however, make the formation-energy prediction equal to the input features by construction.
full rationale
After walking the derivation chain, there is no self-definitional circularity: the recursion coefficients are computed from tight-binding Hamiltonians downfolded from DFT dimer eigenstates (ref. 23) and from BOPfox (ref. 24), not fitted to the 82 formation-energy differences. The AtomF features are elemental properties, and the target Delta E_AB enters only as the supervised output. The heavy use of the authors' own BOPfox and TB parameter references is self-citation, but those parameters are external, published inputs whose assumptions do not include the target values, so under the stated rules they count as independent evidence and do not by themselves raise the circularity score. The two flagged steps are statistical-leakage issues in the evaluation protocol: feature selection on the complete dataset and target-based outlier removal before splits. These make the reported held-out RMSEs optimistically biased and weaken the quantitative comparison to Ghiringhelli et al., but they do not reduce the model output to the input by construction. The directional claim that bond-informed recursion features add predictive information may survive a nested cross-validation, so the appropriate score is moderate rather than 6 or higher.
Assumptions & free parameters
free parameters (4)
- Recursion order N =
4 on D1, 6 on D2
- SFS-selected feature subset =
varies per feature set
- KRR hyperparameters =
from GridSearchCV
- Ensemble weight factors =
30% sub-test, 70% validation, inverse MSE
assumptions (5)
- domain assumption Tight-binding two-center Hamiltonians downfolded from DFT dimer eigenstates (Jenke et al. 2021) transfer to the RS and ZB solids.
- domain assumption A Vegard-scaled ideal prototype cell, without relaxation, is an adequate geometry for recursion feature computation.
- domain assumption A minimal sp3 basis (one s, three p orbitals) averaged via Eq. (2) captures the DOS features relevant to phase stability.
- domain assumption The DFT formation enthalpies of Ghiringhelli et al. are ground truth for the target.
- standard math Lanczos tridiagonalization produces recursion coefficients whose continued fraction exactly represents the local DOS in the infinite-band limit.
Cite this review
Pith. "Pith review of Machine-learning octet $AB$-type binary compounds across chemical space with domain knowledge of the interatomic bond." pith.science (2026). https://pith.science/paper/ZQ2BU3YM
@misc{pith2026260809257,
author = {Pith},
title = {Pith review of: Machine-learning octet $AB$-type binary compounds across chemical space with domain knowledge of the interatomic bond},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZQ2BU3YM}},
note = {Machine review of arXiv:2608.09257}
}
abstract
The prediction of the structural stability of octet $AB$-type binary compounds is a classical materials informatics problem. The challenge is to capture the relative stability of 4-fold coordinated atoms in zincblende ($\beta$-ZnS) structure and 6-fold coordinated atoms in rocksalt (NaCl) structure, modulated by charge transfer and atomic-size differences. Previous structure maps and machine-learning approaches used atomic features such as valence-electron count, ionization potential and atomic radii, using either physical intuition or symbolic regression. Here, we demonstrate that explicitly incorporating the domain knowledge of the interatomic bonds can significantly and systematically improve the prediction of $\beta$-ZnS/NaCl stability. We encode this bonding information through a coarse-grained representation of the local electronic structure obtained by a recursive solution of a tight-binding bond model. The underlying pairwise Hamiltonians are taken from downfolded eigenstates of density-functional theory calculations for diatomic molecules and thereby include domain knowledge of the bond between specific $A-B$ pairs. The benefit of this description is demonstrated with an ensemble of independently trained Kernel Ridge or symbolic regression models combined with sequential feature selection. The obtained models are compared to a previous symbolic-regression model using the same set of \emph{ab initio} calculations for octet binaries as training data. We find a significant improvement in the prediction of the formation energy difference of $AB$ compounds as compared to previous works and demonstrate that an increasing amount of bond-informed recursion features improves the predictive accuracy.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Pilania, J
G. Pilania, J. E. Gubernatis, and T. Lookman, Classification of octet AB-type binary compounds using dynamical charges: A materials informatics perspective, Scientific Reports5, 17504 (2015)
2015
-
[2]
Oliynyk, L
A. Oliynyk, L. Adutwum, J. Harynuk, and A. Mar, Classifying crystal structures of binary compounds AB through cluster resolution feature selection and support vector machine analysis, Chem. Mater.28, 6672 (2016)
2016
-
[3]
L. M. Ghiringhelli, J. Vybiral, S. V. Levchenko, C. Draxl, and M. Scheffler, Big data of materials science: Critical role of the descriptor, Phys. Rev. Lett.114, 105503 (2015)
2015
-
[4]
Bialon, T
A. Bialon, T. Hammerschmidt, and R. Drautz, Three- parameter crystal-structure prediction for sp-d valent compounds, Chem. Mater.28, 2550 (2016)
2016
-
[5]
Isayev, D
O. Isayev, D. Fourches, E. Muratov, C. Oses, K. Rasch, A. Tropsha, and S. Curtarolo, Materials cartography: Representing and mining materials space using structural and electronic fingerprints, Chem. Mater.27, 735 (2015)
2015
-
[6]
Kuban, S
M. Kuban, S. Rigamonti, M. Scheidgen, and C. Draxl, Density-of-states similarity descriptor for unsupervised learning from materials data, Sci. Data9, 646 (2022)
2022
-
[7]
A. Naik, C. Ertural, N. Dhamrait, P. Benner, and J. George, A quantum-chemical bonding database for solid-state materials, Sci. Data10, 610 (2023)
2023
-
[8]
Karamad, R
M. Karamad, R. Magar, Y. Shi, S. Siahrostami, I. Gates, and A. Barati Farimani, Orbital graph convolutional neural network for material property prediction, Phys. Rev. Mater.4, 093801 (2020)
2020
Show all 37 references
-
[9]
Zadoks, A
A. Zadoks, A. Marrazzo, and N. Marzari, Spectral operator representations, npj Comput. Mater.10, 278 (2024)
2024
-
[10]
Haydock, Recursive solution of the Schr¨ odinger equation, Comput
R. Haydock, Recursive solution of the Schr¨ odinger equation, Comput. Phys. Commun.20, 11 (1980)
1980
-
[11]
Haydock, Recursive solution of Schr¨ odinger’s equation, Sol
R. Haydock, Recursive solution of Schr¨ odinger’s equation, Sol. Stat. Phys.35, 215 (1980)
1980
-
[12]
Cryot-Lackmann, On the electronic structure of liquid transition metals, Adv
F. Cryot-Lackmann, On the electronic structure of liquid transition metals, Adv. Phys16, 393 (1967)
1967
-
[13]
Ducastelle and F
F. Ducastelle and F. Cyrot-Lackmann, Moments developments and their application to the electronic charge distribution of d bands, J. Phys. Chem. Solids 31, 1295 (1970)
1970
-
[14]
Ducastelle and F
F. Ducastelle and F. Cyrot-Lackmann, Moments developments - II. Application to the crystalline structures and the stacking fault energies of transition metals, J. Phys. Chem. Solids32, 285 (1971)
1971
-
[15]
Turchi, G
P. Turchi, G. Treglia, and F. Ducastelle, Electronic structure and phase stability of A15 transition metals and alloys, J. Phys.: Met. Phys.13, 2543 (1983)
1983
-
[16]
Bieber, F
A. Bieber, F. Ducastelle, F. Gautier, G. Treglia, and R. Turchi, Electronic structure and relative stabilities of 11 L12 and D0 22 ordered structures occuring in transition metal alloys, Solid State Commun.45, 585 (1983)
1983
-
[17]
Seiser, T
B. Seiser, T. Hammerschmidt, A. N. Kolmogorov, R. Drautz, and D. G. Pettifor, Theory of structural trends within 4d and 5d transition metals topologically close-packed phases, Phys. Rev. B83, 224116 (2011)
2011
-
[18]
Hammerschmidt, A
T. Hammerschmidt, A. Ladines, J. Koßmann, and R. Drautz, Crystal-structure analysis with moments of the density-of-states: Application to intermetallic topologically close-packed phases, Crystals6, 18 (2016)
2016
-
[19]
Jenke, A
J. Jenke, A. P. Subramanyam, M. Densow, T. Hammerschmidt, D. G. Pettifor, and R. Drautz, Electronic structure based descriptor for characterizing local atomic environments, Phys. Rev. B98, 1 (2018)
2018
-
[20]
Sutton, L
C. Sutton, L. M. Ghiringhelli, T. Yamamoto, Y. Lysogorskiy, L. Blumenthal, T. Hammerschmidt, J. R. Golebiowski, X. Liu, A. Ziletti, and M. Scheffler, Crowd-sourcing materials-science challenges with the NOMAD 2018 Kaggle competition, npj Comput. Mater. 5, 1 (2019)
2019
-
[21]
D¨ osinger, T
C. D¨ osinger, T. Hammerschmidt, O. Peil, D. Scheiber, and L. Romaner, Descriptors based on the density of states for efficient machine learning of grain-boundary segregation energies, Comput. Mater. Sci.247, 113493 (2025)
2025
-
[22]
Forti, A
M. Forti, A. Malakhova, Y. Lysogorskiy, W. Zhang, J.-C. Crivello, J.-M. Joubert, R. Drautz, and T. Hammerschmidt, Data-efficient machine-learning of complex Fe-Mo intermetallics using domain knowledge of chemistry and crystallography, npj Comput. Mater. 12, 161 (2026)
2026
-
[23]
Jenke, A
J. Jenke, A. N. Ladines, T. Hammerschmidt, D. G. Pettifor, and R. Drautz, Tight-binding bond parameters for dimers across the periodic table from density- functional theory, Phys. Rev. Materials5, 023801 (2021)
2021
-
[24]
Hammerschmidt, B
T. Hammerschmidt, B. Seiser, M. E. Ford, A. N. Ladines, S. Schreiber, N. Wang, J. Jenke, Y. Lysogorskiy, C. Teijeiro, M. Mrovec, M. Cak, E. R. Margine, D. G. Pettifor, and R. Drautz, BOPfox program for tight- binding and analytic bond-order potential calculations, Comput. Phys...
2019
-
[25]
Lanczos, An iterative method for the solution of the eigenvalue problem of linear differential and integral, J
C. Lanczos, An iterative method for the solution of the eigenvalue problem of linear differential and integral, J. Res. Natl. Bur. Stand.45(1950)
1950
-
[26]
Aoki, Rapidly convergent bond order expansion for atomistic simulations, Phys
M. Aoki, Rapidly convergent bond order expansion for atomistic simulations, Phys. Rev. Lett.71, 3842 (1993)
1993
-
[27]
Gehrmann, D
J. Gehrmann, D. G. Pettifor, A. N. Kolmogorov, M. Reese, M. Mrovec, C. Els¨ asser, and R. Drautz, Reduced tight-binding models for elemental si and n, and ordered binary si-n systems, Phys. Rev. B91, 054109 (2015)
2015
-
[28]
Vegard, Die Konstitution der Mischkristalle und die Raumf¨ ullung der Atome, Zeitschrift f¨ ur Physik5, 17 (1921)
L. Vegard, Die Konstitution der Mischkristalle und die Raumf¨ ullung der Atome, Zeitschrift f¨ ur Physik5, 17 (1921)
1921
-
[29]
A. R. Denton, Vegard’s law, Physical Review A43, 3161 (1991)
1991
-
[30]
F. T. Liu, K. M. Ting, and Z.-H. Zhou, Isolation-based anomaly detection, ACM Transactions on Knowledge Discovery from Data6, 3:1 (2012)
2012
-
[31]
Raschka, Mlxtend: Providing machine learning and data science utilities and extensions to python’s scientific computing stack, Journal of Open Source Software3, 638 (2018)
S. Raschka, Mlxtend: Providing machine learning and data science utilities and extensions to python’s scientific computing stack, Journal of Open Source Software3, 638 (2018)
2018
-
[32]
Pudil, J
P. Pudil, J. Novoviˇ cov´ a, and J. Kittler, Floating search methods in feature selection, Pattern Recognition Letters 15, 1119 (1994)
1994
-
[33]
F. J. Ferri, P. Pudil, M. Hatef, and J. Kittler, Comparative study of techniques for large-scale feature selection, inMachine Intelligence and Pattern Recognition, Vol. 16 (Elsevier, 1994) pp. 403–413
1994
-
[34]
Ouyang, S
R. Ouyang, S. Curtarolo, E. Ahmetcik, M. Scheffler, and L. M. Ghiringhelli, SISSO: A compressed-sensing method for identifying the best low-dimensional descriptor in an immensity of offered candidates, Phys. Rev. Mater.2, 083802 (2018)
2018
-
[35]
T. A. R. Purcell, M. Scheffler, and L. M. Ghiringhelli, Recent advances in the SISSO method and their implementation in the SISSO++ code, The Journal of Chemical Physics159, 114110 (2023)
2023
-
[36]
Machine-learning octet AB-type binary compounds across chemical space with domain knowledge of the interatomic bond
L. Breiman, Random forests, Machine learning45, 5 (2001). Supplementary material for article “Machine-learning octet AB-type binary compounds across chemical space with domain knowledge of the interatomic bond” Rohan Kumar, 1,∗ Mariano Forti, 1,† Aakash A. Naik, 2, 3 Luca M. G...
2001
-
[100]
F AIRmat
The number of residuals considered, (n residuals), is set to 10. Finally, the complexity of the constructed descriptors is limited by restricting the maximum number of successive mathematical operations (the maximum rung) to 2. Since a descriptor is constructed anew within eve...
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.