Modelling magnetic material properties with uncertainty-aware neural networks
Pith reviewed 2026-06-27 09:15 UTC · model grok-4.3
The pith
Uncertainty quantification via neural networks makes magnetic property predictions more trustworthy and transfers to new tasks.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The authors demonstrate that uncertainty quantification not only enhances the trustworthiness of predictions but is also transferable across different modeling tasks: standard models for intrinsic magnetic properties and a graph neural network for coercivity both benefit from Gaussian negative log-likelihood loss and dropout-based Bayesian approximation.
What carries the argument
Gaussian negative log-likelihood loss combined with dropout-based Bayesian approximation to produce predictive uncertainty estimates in neural networks.
If this is right
- Researchers can use the uncertainty values to flag and set aside low-confidence predictions during material screening.
- The same uncertainty machinery can be reused when moving from simple property models to graph-based microstructure models without new calibration.
- Out-of-distribution cases in compositional design spaces become detectable rather than hidden.
- Model outputs for coercivity gain an accompanying reliability score that was previously unavailable.
Where Pith is reading between the lines
- The transferability could support modular uncertainty components that plug into multiple material prediction pipelines.
- High-uncertainty predictions could guide targeted experiments in an active-learning loop for permanent magnets.
- Similar uncertainty handling might extend to other physical properties where data sparsity and out-of-distribution needs are common.
Load-bearing premise
The chosen loss and dropout methods produce well-calibrated uncertainty estimates that remain useful when models encounter new material structures.
What would settle it
A collection of previously unseen magnetic material compositions where the model's reported uncertainty shows no correlation with its actual prediction error.
Figures
read the original abstract
Machine learning is increasingly applied to accelerate the discovery of novel materials by exploring large compositional and structural design spaces. Yet, the scarcity of high-quality data and the frequent need for out-of-distribution prediction introduce substantial uncertainty, making the assessment of model reliability essential. In this work, we investigate uncertainty quantification as a means to evaluate model confidence in the context of permanent magnet research. In a first study, we benchmark classical and modern machine learning models for predicting intrinsic magnetic properties, focusing on the quality of their uncertainty estimates. We apply Gaussian negative log-likelihood loss and dropout-based Bayesian approximation as practical strategies for estimating predictive uncertainty. In a second study, we transfer these architectural features for uncertainty estimation to a more complex task: predicting coercivity from microstructural information using a graph neural network. Together, these studies demonstrate that uncertainty quantification not only enhances the trustworthiness of predictions but is also transferable across different modeling tasks.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper investigates uncertainty quantification (UQ) in machine learning for magnetic materials, with two studies. The first benchmarks classical and modern ML models on intrinsic magnetic properties using Gaussian negative log-likelihood loss and dropout-based Bayesian approximation to estimate predictive uncertainty. The second transfers these UQ techniques to a graph neural network task predicting coercivity from microstructural data. The central claim is that UQ enhances the trustworthiness of predictions and is transferable across modeling tasks in permanent magnet research.
Significance. If the empirical results demonstrate well-calibrated uncertainties that improve decision-making in data-scarce and out-of-distribution settings, the work could offer practical tools for reliable ML-assisted materials discovery. The transferability across intrinsic-property regression and GNN-based microstructure tasks would be a notable contribution if supported by quantitative evidence such as calibration metrics and ablation studies.
major comments (2)
- [Abstract] Abstract: the central claim that the two studies 'demonstrate' enhanced trustworthiness and transferability of UQ cannot be evaluated, as the provided text contains no results, validation metrics, error bars, calibration plots, or ablation studies comparing models with and without UQ.
- [Abstract] The weakest assumption—that Gaussian NLL loss combined with dropout produces well-calibrated uncertainties useful for OOD predictions in magnetic-property modeling—is stated but not tested in the visible material; without explicit calibration or OOD experiments, the transferability claim remains ungrounded.
Simulated Author's Rebuttal
We thank the referee for their comments on the abstract. We address each point below and note that the full manuscript contains the supporting results, metrics, and experiments referenced in the studies.
read point-by-point responses
-
Referee: [Abstract] Abstract: the central claim that the two studies 'demonstrate' enhanced trustworthiness and transferability of UQ cannot be evaluated, as the provided text contains no results, validation metrics, error bars, calibration plots, or ablation studies comparing models with and without UQ.
Authors: The abstract serves as a concise summary and conventionally omits detailed numerical results, plots, or metrics, which appear in the main text. The manuscript reports validation metrics, error bars, calibration plots, and ablation studies comparing models with and without UQ for both the intrinsic-property benchmarks and the GNN coercivity task. We will revise the abstract to more precisely indicate that the demonstrations rest on the empirical evidence detailed in the results sections. revision: yes
-
Referee: [Abstract] The weakest assumption—that Gaussian NLL loss combined with dropout produces well-calibrated uncertainties useful for OOD predictions in magnetic-property modeling—is stated but not tested in the visible material; without explicit calibration or OOD experiments, the transferability claim remains ungrounded.
Authors: The full manuscript contains explicit calibration assessments and out-of-distribution experiments for uncertainties obtained with Gaussian negative log-likelihood loss and dropout-based approximation. These are presented for the classical and modern ML models on intrinsic magnetic properties and transferred to the graph neural network microstructure task. We will revise the abstract wording to better reflect the presence of these tests in the manuscript. revision: yes
Circularity Check
No significant circularity detected
full rationale
The paper presents an empirical study applying Gaussian NLL loss and dropout-based uncertainty estimation first to intrinsic magnetic property prediction and then to a GNN-based coercivity task. No equations, fitted parameters, or derivation chains are described in the provided text. The central claim of transferability is an experimental observation rather than a mathematical reduction to inputs by construction. No self-citations, ansatzes, or uniqueness theorems appear as load-bearing elements. The work is therefore self-contained against external benchmarks with no detectable circular steps.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
J. Lai, A. Bolyachkin, N. Terada, S. Dieb, X. Tang, T. Ohkubo, H. Sepehri- Amin, K. Hono, Machine learning assisted development of Fe2P-type magnetocaloric compounds for cryogenic applications, Acta Materialia 232 (2022) 117942.doi:10.1016/j.actamat.2022.117942. URLhttps://linkinghub.elsevier.com/retrieve/pii/ S1359645422003238
-
[2]
E. Dengina, A. Bolyachkin, H. Sepehri-Amin, K. Hono, Machine Learning Approach for Evaluation of Nanodefects and Magnetic Anisotropy in FePt Granular Films, Scripta Materialia 218 (2022) 114797.doi:10.1016/j.scriptamat.2022.114797. URLhttps://linkinghub.elsevier.com/retrieve/pii/ S1359646222002937
-
[3]
N. Kulesh, A. Bolyachkin, I. Suzuki, Y . Takahashi, H. Sepehri-Amin, K. Hono, Data-driven optimization of FePt heat-assisted magnetic recording media accelerated by deep learning TEM image segmentation, Acta Materi- alia 255 (2023) 119039.doi:10.1016/j.actamat.2023.119039. URLhttps://linkinghub.elsevier.com/retrieve/pii/ S1359645423003701
-
[4]
A. Kovacs, J. Fischbacher, H. Oezelt, A. Kornell, Q. Ali, M. Gusenbauer, M. Yano, N. Sakuma, A. Kinoshita, T. Shoji, A. Kato, Y . Hong, S. Grenier, T. Devillers, N. M. Dempsey, T. Fukushima, H. Akai, N. Kawashima, T. Miyake, T. Schrefl, Physics-informed machine learning combining exper- iment and simulation for the design of neodymium-iron-boron permanent...
-
[5]
A. Srinithi, A. Bolyachkin, X. Tang, H. Sepehri-Amin, S. Dieb, A. Saito, T. Ohkubo, K. Hono, Data-driven compositional op- timization of La(Fe,Si)13-based magnetocaloric compounds for cryogenic applications, Scripta Materialia 258 (2025) 116486. doi:10.1016/j.scriptamat.2024.116486. URLhttps://linkinghub.elsevier.com/retrieve/pii/ S1359646224005219
-
[6]
T. Lookman, P. V . Balachandran, D. Xue, R. Yuan, Active learning in materials science with emphasis on adaptive sampling using uncertainties for targeted design, npj Computational Materials 5 (1) (2019) 21.doi: 10.1038/s41524-019-0153-8. URLhttps://www.nature.com/articles/s41524-019-0153-8
-
[7]
V . Korolev, I. Nevolin, P. Protsenko, A universal similarity based approach for predictive uncertainty quantification in materials science, Scientific Re- ports 12 (1) (2022) 14931.doi:10.1038/s41598-022-19205-5. URLhttps://www.nature.com/articles/s41598-022-19205-5
-
[8]
K. Tran, W. Neiswanger, J. Yoon, Q. Zhang, E. Xing, Z. W. Ulissi, Methods for comparing uncertainty quantifications for material property predic- tions, Machine Learning: Science and Technology 1 (2) (2020) 025006. doi:10.1088/2632-2153/ab7e1a. URLhttps://iopscience.iop.org/article/10.1088/2632-2153/ ab7e1a
-
[9]
I. G. De Moraes, J. Fischbacher, Y . Hong, C. Naud, H. Okuno, A. Masseboeuf, T. Devillers, T. Schrefl, N. M. Dempsey, Nanofab- rication, characterisation and modelling of soft-in-hard FeCo–FePt magnetic nanocomposites, Acta Materialia 274 (2024) 119970. doi:10.1016/j.actamat.2024.119970. URLhttps://linkinghub.elsevier.com/retrieve/pii/ S1359645424003227
-
[10]
C. E. Rasmussen, C. K. I. Williams, Gaussian Processes for Machine Learn- 19 ing, Adaptive Computation and Machine Learning, MIT Press, Cambridge, Mass, 2006
2006
-
[11]
Machine Learning45, 5–32 (2001)
L. Breiman, Random Forests, Machine Learning 45 (1) (2001) 5–32.doi: 10.1023/A:1010933404324. URLhttp://link.springer.com/10.1023/A:1010933404324
-
[12]
Y . Gal, Z. Ghahramani, Dropout as a Bayesian Approximation: Represent- ing Model Uncertainty in Deep Learning (Oct. 2016).arXiv:1506.02142. URLhttp://arxiv.org/abs/1506.02142
Pith/arXiv arXiv 2016
-
[13]
A. Kendall, Y . Gal, What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? (Oct. 2017).arXiv:1703.04977. URLhttp://arxiv.org/abs/1703.04977
Pith/arXiv arXiv 2017
- [14]
-
[15]
Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
A. Kendall, Y . Gal, R. Cipolla, Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics (Apr. 2018).arXiv: 1705.07115,doi:10.48550/arXiv.1705.07115. URLhttp://arxiv.org/abs/1705.07115
work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.1705.07115 2018
-
[16]
R. Green, M. Rowe, A. Polleri, MACEst: The reliable and trustworthy Model Agnostic Confidence Estimator (Sep. 2021).arXiv:2109.01531, doi:10.48550/arXiv.2109.01531. URLhttp://arxiv.org/abs/2109.01531
-
[17]
Chollet, Keras (2015)
F. Chollet, Keras (2015). URLhttps://keras.io
2015
-
[18]
Geron, Hands-On Machine Learning with Scikit-Learn, Keras & Tensor- Flow, 2nd Edition, 2019
A. Geron, Hands-On Machine Learning with Scikit-Learn, Keras & Tensor- Flow, 2nd Edition, 2019
2019
-
[19]
D. Nix, A. Weigend, Estimating the mean and variance of the target prob- ability distribution, in: Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN’94), IEEE, Orlando, FL, USA, 1994, pp. 55–60 vol.1.doi:10.1109/ICNN.1994.374138. URLhttp://ieeexplore.ieee.org/document/374138/ 20
-
[20]
B. Lakshminarayanan, A. Pritzel, C. Blundell, Simple and Scalable Predic- tive Uncertainty Estimation using Deep Ensembles (Nov. 2017).arXiv: 1612.01474. URLhttp://arxiv.org/abs/1612.01474
Pith/arXiv arXiv 2017
-
[21]
Nguyen, Why Uncertainty Matters in Deep Learning and How to Estimate It « Searching Gradients (Jan
H. Nguyen, Why Uncertainty Matters in Deep Learning and How to Estimate It « Searching Gradients (Jan. 2020). URLhttps://everyhue.me/posts/why-uncertainty-matters/
2020
-
[22]
PyTorch: An Imperative Style, High-Performance Deep Learning Library
A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Köpf, E. Yang, Z. De- Vito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, S. Chintala, PyTorch: An Imperative Style, High-Performance Deep Learn- ing Library (Dec. 2019).arXiv:1912.01703,doi:10.48550/arXiv. ...
work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv 2019
-
[23]
N. Dewolf, B. D. Baets, W. Waegeman, Valid prediction intervals for re- gression problems, Artificial Intelligence Review 56 (1) (2023) 577–613. doi:10.1007/s10462-022-10178-5. URLhttps://link.springer.com/10.1007/s10462-022-10178-5
-
[24]
Pedregosa, G
F. Pedregosa, G. Varoquaux, A. Gramfort, V . Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V . Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, É. Duchesnay, Scikit-learn: Machine Learning in Python, Journal of Machine Learning Research 12 (85) (2011) 2825–2830. URLhttp://jmlr.org/papers/v12/pedregosa11a.html
2011
-
[25]
Bradbury, R
J. Bradbury, R. Frostig, P. Hawkins, M. James Johnson, C. Leary, D. Maclau- rin, JAX: Composable transformations of {P}ython+{N}um{P}y programs (2018). URLhttp://github.com/jax-ml/jax
2018
-
[26]
Pernot, Confidence curves for UQ validation: Probabilistic reference vs
P. Pernot, Confidence curves for UQ validation: Probabilistic reference vs. oracle (Dec. 2022).arXiv:2206.15272,doi:10.48550/arXiv.2206. 15272. URLhttp://arxiv.org/abs/2206.15272 21
-
[27]
J. Park, C. Yoon, C. Park, J. Ahn, Kernel Methods for Radial Transformed Compositional Data with Many Zeros 162 (2022). URLhttps://proceedings.mlr.press/v162/park22d.html
2022
-
[28]
H. Moustafa, A. Kovacs, J. Fischbacher, M. Gusenbauer, Q. Ali, L. Breth, T. Schrefl, H. Oezelt, Graph neural networks to predict co- ercivity and maximum energy product of hard magnetic microstructures, Journal of Magnetism and Magnetic Materials 634 (2025) 173594. doi:10.1016/j.jmmm.2025.173594. URLhttps://linkinghub.elsevier.com/retrieve/pii/ S0304885325008261
-
[29]
R. Quey, P. Dawson, F. Barbe, Large-scale 3D random polycrystals for the finite element method: Generation, meshing and remeshing, Computer Methods in Applied Mechanics and Engineering 200 (17-20) (2011) 1729–1745.doi:10.1016/j.cma.2011.01.002. URLhttps://linkinghub.elsevier.com/retrieve/pii/ S004578251100003X
-
[30]
H. Moustafa, A. Kovacs, J. Fischbacher, M. Gusenbauer, Q. Ali, L. Breth, Y . Hong, W. Rigaut, T. Devillers, N. M. Dempsey, T. Schrefl, H. Oezelt, Reduced order model for hard magnetic films, AIP Advances 14 (2) (2024) 025001.doi:10.1063/9.0000816. URLhttps://pubs.aip.org/adv/article/14/2/025001/3261431/ Reduced-order-model-for-hard-magnetic-films
-
[31]
M. Fey, J. E. Lenssen, Fast Graph Representation Learning with PyTorch Geometric, version Number: 3 (2019).doi:10.48550/ARXIV.1903. 02428. URLhttps://arxiv.org/abs/1903.02428
work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.1903 2019
-
[32]
M. Dai, M. F. Demirel, Y . Liang, J.-M. Hu, Graph neural networks for an accurate and interpretable prediction of the properties of polycrystalline materials, npj Computational Materials 7 (1) (2021) 103.doi:10.1038/ s41524-021-00574-w. URLhttps://www.nature.com/articles/s41524-021-00574-w 22 Figure 5:AEvaluation plots of the Gaussian process model predic...
2021
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.