Distillation of atomistic foundation models across architectures and chemical domains
read the original abstract
Machine-learned interatomic potentials have transformed computational research in the physical sciences. Recent atomistic `foundation' models have changed the field yet again: trained on many different chemical elements and domains, these potentials are widely applicable, but comparably slow and resource-intensive to run. Here we show how distillation via synthetic data can be used to cheaply transfer knowledge from atomistic foundation models to a range of different architectures, unlocking much smaller, more efficient potentials. We demonstrate speed-ups of $> 10\times$ by distilling from one graph-network architecture into another, and $> 100\times$ by leveraging the atomic cluster expansion framework. We showcase applicability across chemical and materials domains: from liquid water to hydrogen under extreme conditions; from porous silica and a hybrid halide perovskite solar-cell material to modelling organic reactions. Our work shows how distillation can support the routine and computationally efficient use of current and future atomistic foundation models in real-world scientific research.
This paper has not been read by Pith yet.
Forward citations
Cited by 5 Pith papers
-
Lang2MLIP: End-to-End Language-to-Machine Learning Interatomic Potential Development with Autonomous Agentic Workflows
Lang2MLIP is an LLM multi-agent framework that automates end-to-end development of machine learning interatomic potentials from natural language input for heterogeneous materials systems.
-
Robust and Interpretable Adaptation of Equivariant Materials Foundation Models via Sparsity-promoting Fine-tuning
Sparsity-promoting fine-tuning adapts equivariant materials foundation models by selectively updating ~3% of parameters to match full fine-tuning on molecular and crystalline benchmarks while revealing interpretable p...
-
Distilling first-principles accuracy into compact machine learning potentials for condensed-phase chemistry
Distilled compact MLIPs from transfer-learned teachers reproduce observables more reliably than same-size models trained directly and enable practical PIMD umbrella sampling of water dissociation at TiO2 interface wit...
-
Data-Driven Thermal and Mechanical Modeling of Defective Covalent Organic Frameworks
QCOF ML potentials tuned on COF data outperform general MACE models for defective systems and reveal higher thermal defect sensitivity in CTF-1 versus COF-LZU1 with nearly invariant low-strain mechanics.
-
Six Open Questions in Machine-Learned Interatomic Potential Foundation Models
This perspective article develops a definition of foundational MLIPs and poses six open questions that the authors believe will define future research in machine-learned interatomic potentials.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.