Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

XxaCT-NN: Structure Agnostic Multimodal Learning for Materials Science

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a model using only elemental composition and X-ray diffraction, with no crystal-structure input, predicts formation energy with 28.2 meV/atom MAE and crystal system with 97.2% accuracy on a 5-million-compound…

desk verdict A credible structure-free multimodal baseline with a genuinely new masked-XRD pretraining idea, undermined by an uncontrolled comparison to ALIGNN and wrong speedup arithmetic. read the letter →

arxiv 2507.01054 v1 pith:CD6OLXC5 submitted 2025-06-27 cs.LG cond-mat.mtrl-scics.AI

Jithendaraa Subramanian , Linda Hung , Daniel Schweigert , Santosh Suram , Weike Ye This is my paper · ORCID
classification cs.LGcond-mat.mtrl-scics.AI
keywords multimodallearningmaterialsscienceX-raydiffractionelementalcompositioncross-attentionmaskedXRDmodelingself-supervisedpretrainingstructure-agnosticmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to establish that experimentally accessible inputs—elemental composition and X-ray diffraction (XRD)—can replace crystal structure as the basis for high-accuracy materials property prediction. Training a cross-attention bimodal transformer on the 5-million-entry Alexandria dataset, it reports a formation-energy MAE of 28.2 meV/atom and crystal-system accuracy of 97.2%, approaching the 16 meV/atom of structure-based GNNs on the same data. The authors introduce masked XRD modeling (MXM), a self-supervised objective that reconstructs masked XRD intensities, and show that MXM plus contrastive alignment speeds up fine-tuning by up to 4.2× and improves latent representations. A scaling study indicates multimodal models improve more steeply with dataset size than unimodal ones. If these results hold, structure-free foundation models grounded in measurement data become a realistic path for experimental materials discovery.

What carries the argument

The load-bearing mechanism is cross-attention fusion: a 12-layer transformer decoder treats composition-token embeddings as keys and values and XRD-token embeddings as queries, so each XRD segment can dynamically attend to relevant compositional context. The second novel component is masked XRD modeling (MXM), in which 5% of the 17 XRD tokens are replaced with a [MASK] token and the model must reconstruct the masked 250-dimensional intensity vectors from the fused embedding, a regression loss that forces the fusion module to learn localized peak shape and position. Contrastive alignment (CLIP-style, with a learnable temperature) operates on the [CLS] embeddings of the two modalities and aligns paired composition–XRD representations before fusion. Together these objectives update both encoders and the fusion module, which the authors attribute as the reason MXM helps more than contrastive loss alone.

What would settle it

Take a set of real XRD measurements with known compositions and independently known formation energies, apply the XxaCT-NN trained on Alexandria, and compare predicted energies and crystal systems to the measured values; if error rises substantially above the reported 28.2 meV/atom (or if adding realistic background/noise to simulated patterns degrades accuracy sharply), the experimentally grounded claim is falsified.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that fusion of composition and XRD via cross-attention yields predictions that are close to structure-based models without ever seeing atomic coordinates. The best single-task XxaCT-NN model reaches a formation-energy MAE of 28.2 meV/atom and 96.8% crystal-system accuracy (97.2% in the multi-task setting), against the reproduced 147.6 meV/atom of the prior XRD+composition baseline and the 16 meV/atom reference for a structure-based GNN on different splits of the same dataset. The paper further claims that masked XRD modeling (MXM)—a regression analog of masked language modeling applied to 5% of XRD tokens—plus contrastive alignment give faster convergence (up to 4.2×), better test accuracy, and better crystal-system-aligned latent clusters (silhouette 0.50 vs 0.42 without pretraining). Finally, the scaling experiment shows bimodal error falling as $L = 0.07 \cdot D^{-0.335}$ versus $L = 0.14 \cdot D^{-0.046}$ for composition alone, so the advantage of the second modality grows with data.

Load-bearing premise

The entire experimental-grounding claim rests on simulated XRD stick patterns—computed from DFT crystal structures and smeared with a Gaussian of σ = 0.1—being a faithful enough proxy for real laboratory X-ray diffraction that learned patterns transfer, yet the paper's Limitations section states the model does not explicitly account for the domain shift between simulated and experimental data, including background, instrument artifacts, and peak shift.

Editorial extensions

If this is right

  • Labs that have composition and XRD but not solved structures could obtain property predictions comparable to structure-based models, skipping the structure-determination step.
  • Self-supervised pretraining on unlabeled XRD–composition pairs is a viable route to faster convergence and better representations, so collecting more paired experimental data should compound gains.
  • Scaling behavior implies multimodal materials models will profit more from bigger datasets than unimodal ones, favoring continued database growth.
  • A pretrained bimodal model transfers to unseen targets: frozen-encoder band-gap MAE of 0.063 eV beats the best unimodal XRD model by roughly 30%.
  • Adapting masked-language-style objectives to characterization data (XRD) is a template for other measurement modalities.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A likely near-term extension is fine-tuning or calibration on real experimental XRD patterns; if the domain gap can be bridged with modest target data, the same architecture becomes deployable in automated synthesis-and-characterization loops.
  • MXM's focus on reconstructing peak positions and shapes suggests augmentation strategies—random peak shifts, added backgrounds, lower resolution—during pretraining could make the model robust to the experimental noise sources the paper lists as limitations.
  • The framework is not inherently XRD-specific: the same cross-attention layout could fuse composition with other readily measured characterizations (e.g., XRF, EDS, Raman) to build a general structure-free materials foundation model.
  • The demonstrated scaling exponent difference implies that unlabeled experimental XRD–composition pairs, even without property labels, are valuable pretraining data; this suggests a data-collection strategy for the community.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces XxaCT-NN, a structure-free multimodal framework that predicts formation energy, crystal system, and band gap from elemental composition and simulated X-ray diffraction patterns, trained on the 5-million-sample Alexandria dataset. The architecture combines a CrabNet-style composition encoder, a transformer-based XRD encoder, and a cross-attention fusion module. The authors propose masked XRD modeling (MXM) and combine it with contrastive alignment as self-supervised pretraining objectives, reporting faster convergence (up to 4.2x), improved accuracy, and better representation quality. They also report a formation-energy MAE of 28.2 meV/atom and crystal-system accuracy of 97.2%, claiming this approaches the 16 meV/atom of a structure-based ALIGNN model. The paper closes with scaling experiments suggesting multimodal models benefit more from larger datasets than unimodal composition-only models.

Significance. If the empirical claims hold, the contribution is significant: it is one of the largest demonstrations of structure-agnostic multimodal learning for materials, and the MXM pretraining objective is a sensible adaptation of masked modeling to XRD. The paper deserves credit for reproducing a prior PXRDPIAYN baseline on the same dataset, for a clear and honest Limitations section, and for framing the simulated-to-experimental gap as an open challenge rather than claiming it is solved. However, the significance is currently conditional: the headline comparison to ALIGNN uses a non-comparable split, the speedup arithmetic is internally inconsistent, and the pretraining benefits are reported without error bars. These issues directly affect the abstract's strongest quantitative claims and must be resolved before the paper can be evaluated as a reliable advance.

major comments (5)
  1. [Section 4.1, Table 1] The central claim that XxaCT-NN 'approaches the SOTA performance (16 meV/atom) of structure-based GNNs trained on the same Alexandria dataset' is not supported by the evidence presented. Table 1 explicitly labels the ALIGNN entry as 'on different splits,' and MAE values are defined with respect to a specific test split. The 12 meV/atom gap between 28.2 and 16 meV/atom could be due to split composition, label version, or evaluation protocol rather than model capability. The paper should either reproduce ALIGNN under the same split and preprocessing, or reword the abstract and Section 4.1 to restrict the SOTA comparison to the controlled internal baselines.
  2. [Section 4.2, convergence speedups] The speedup arithmetic is internally inconsistent. The text states that the contrastive + MXM model reaches a threshold at 3,000 iterations while the baseline requires over 12,000, yielding 4.2x, but 12,000/3,000 = 4.0. More seriously, the reported MXM-only and contrastive-only speedups of 1.8x and 1.2x at approximately 2,300 and 3,500 iterations imply a baseline of roughly 4,200 iterations, not 12,000. These numbers cannot all be correct. The abstract's 'up to 4.2x speedup' claim and the convergence discussion in Section 4.2 therefore need a corrected and consistent set of iteration-to-threshold measurements.
  3. [Section 4.2, Table 2] The pretraining comparisons rest on single training runs with no error bars or repeated-seed statistics. For example, the best MAE values 45.7, 44.49, 43.48, and 43.82 meV differ by 1-2 meV, which is typically within run-to-run noise for transformer-scale models, and the conclusion that MXM outperforms contrastive pretraining depends on this small margin. The paper should either provide multiple seeds with standard deviations or temper the comparative claims about pretraining strategies.
  4. [Section 4.4, scaling analysis] The scaling claim that 'multimodal performance scales more favorably with dataset size than unimodal baselines' is based on power-law fits L = 0.14*D^-0.046 and L = 0.07*D^-0.335, but the paper does not report the number of data points, the fit quality (e.g., R^2), or any uncertainty on the exponents. With only a few dataset sizes (1M to 4.5M), the extrapolation that gains 'compound at larger data regimes' is not yet supported. The authors should quantify the fit and either add more scale points or soften the abstract's scaling statement.
  5. [Limitations section] The Limitations section explicitly states that the model 'does not explicitly account for the domain shift between simulated and experimental data,' including background, instrument artifacts, and peak shift. In light of this, the abstract's claim that the results 'establish a path toward structure-free, experimentally grounded foundation models' and the introduction's emphasis on practicality for real-world workflows are overstated. No experiment with real measured XRD is presented. The authors should either add a real-XRD validation or substantially qualify the 'experimentally grounded' language.
minor comments (5)
  1. [Section 4.4, text] The sentence 'the models is evaluated on the same test set' contains a subject-verb agreement error and should read 'the models are evaluated.'
  2. [Section 3.2, Eq. (3)] The notation {f_cls, f_1, ..., f_N} for fused embeddings is introduced as corresponding to masked tokens, but the mapping between the masked tokens and the fused embeddings is not fully explicit; please clarify whether f_i is the fusion output at the position of token i and how the [MASK] token is represented.
  3. [Table 1] The transfer MAE column entry '0.063 | 0.1387' mixes two different numbers of decimal places and is not explained in the table caption; please define the notation (e.g., Ef-pretrained | crystal-system-pretrained) and use consistent formatting.
  4. [Section 3.1, Figure 1] The tokens labeled 'C L S' in the schematic are not defined in the main text; if these denote [CLS] tokens, please state this explicitly in the figure caption or Section 3.1.
  5. [Appendix A.2] The description of the PXRDPIAYN baseline adaptation notes small changes to layer dimensions but does not list the exact layer configurations used in the reproduced model; a short table or reference to the code repository would improve reproducibility.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the headline benchmark results are independent empirical evaluations; the only author-overlap citation (UniMat) appears in Related Work and is not load-bearing.

full rationale

The paper's central quantitative claims—28.2 meV/atom Ef MAE, 97.2% crystal-system accuracy, transfer band-gap MAEs, pretraining speedups, and scaling exponents—are measured on a held-out 491,520-entry test split and are not derived from the model's own fitted parameters. The PXRDPIAYN baseline was retrained on the same Alexandria data (147.6 meV/atom), providing an internal control. No equation in the paper defines an output in terms of the quantity it claims to predict; MXM reconstructs masked XRD tokens from unmasked inputs and composition, which is a genuine self-supervised task. The scaling power laws in Section 4.4 (L = 0.07·D^−0.335) are fitted descriptions of the shown data points, not predictions, so they are descriptive rather than circular. The only self-citation with author overlap is UniMat [Ock et al., 2024], cited in Related Work to describe prior concatenation-based fusion on MP20; this does not carry the paper's claims. Two validity caveats, neither circular, should be noted: the ALIGNN 16 meV/atom SOTA figure is qualified as 'on different splits' and is therefore an uncontrolled comparison, and the Limitations section explicitly acknowledges the simulated-to-experimental XRD domain shift, weakening the 'experimentally grounded' framing but not constituting circularity.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

No new physical entities are introduced. The free parameters are data-processing and modeling choices plus the two fitted scaling exponents that underpin the scaling claim. The axioms are domain assumptions about data provenance and split validity.

free parameters (4)
  • Gaussian smearing width sigma = 0.1
    Hand-chosen preprocessing for XRD peak broadening; directly shapes all XRD inputs and therefore the reported accuracies.
  • MXM masking ratio = 5%
    Chosen for the masked XRD modeling objective; no ablation is run against other ratios such as BERT's typical 15%.
  • Unimodal scaling exponent = -0.046
    Power-law exponent fit to four dataset-size points (1M to 4.5M) in Figure 3; used to characterize the unimodal baseline as weakly scaling.
  • Bimodal scaling exponent = -0.335
    Power-law exponent fit to the same four points; this exponent is the quantitative basis for the claim that multimodal models scale more favorably.
assumptions (3)
  • domain assumption Simulated XRD patterns computed from Alexandria crystal structures are treated as valid inputs for the structure-free model.
    The whole premise of 'experimentally grounded' models rests on this, and the paper lists the simulated-to-experimental domain shift as an open limitation.
  • domain assumption The random 90/10 split of Alexandria is leakage-free and representative.
    Compositions and near-duplicate structures may appear in both train and test after random shuffling, which can inflate test accuracy; no deduplication or composition-based split is described.
  • domain assumption DFT formation energies and Pymatgen-derived crystal systems in Alexandria are treated as ground-truth targets.
    The model's accuracy is measured against these labels; errors in DFT or symmetry assignment are inherited.

how reviews work

0 comments
Cite this review

Pith. "Pith review of XxaCT-NN: Structure Agnostic Multimodal Learning for Materials Science." pith.science (2026). https://pith.science/paper/CD6OLXC5

@misc{pith2026250701054,
  author       = {Pith},
  title        = {Pith review of: XxaCT-NN: Structure Agnostic Multimodal Learning for Materials Science},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CD6OLXC5}},
  note         = {Machine review of arXiv:2507.01054}
}
read the original abstract

Recent advances in materials discovery have been driven by structure-based models, particularly those using crystal graphs. While effective for computational datasets, these models are impractical for real-world applications where atomic structures are often unknown or difficult to obtain. We propose a scalable multimodal framework that learns directly from elemental composition and X-ray diffraction (XRD) -- two of the more available modalities in experimental workflows without requiring crystal structure input. Our architecture integrates modality-specific encoders with a cross-attention fusion module and is trained on the 5-million-sample Alexandria dataset. We present masked XRD modeling (MXM), and apply MXM and contrastive alignment as self-supervised pretraining strategies. Pretraining yields faster convergence (up to 4.2x speedup) and improves both accuracy and representation quality. We further demonstrate that multimodal performance scales more favorably with dataset size than unimodal baselines, with gains compounding at larger data regimes. Our results establish a path toward structure-free, experimentally grounded foundation models for materials science.

Figures

Figures reproduced from arXiv: 2507.01054 by the authors.

Figure 1
Figure 1. Schematic illustration of XxaCT-NN , our proposed multimodal framework. XxaCT-NN consists of separate self-attention encoders for composition and XRD inputs, each producing modality-specific embeddings. The embeddings are fused via a cross-attention module and jointly updated for downstream tasks, including formation energy regression and crystal system clas￾sification. Pretraining options include masked XRD modelin… view at source ↗
Figure 2
Figure 2. PCA visualizations of learned latent embeddings colored by crystal system. [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Test set performance as a function of dataset size. XxaCT-NN (orange) has a more negative scaling exponent compared to the uni￾modal model (green). The performance gap widens from 66.2 meV to 97.6 meV as data in￾creases from 1M to 4.5M, demonstrating that multimodal models benefit more from scaling. As shown in [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Crystal system distribution across train and test sets. [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: Visualizing element distribution in Alexandria [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Effect of pretraining on downstream performance and convergence speed. Top row: [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Spectra-Scope : A toolkit for automated and interpretable characterization of material properties from spectral data

    cond-mat.mtrl-sci 2026-03 unverdicted novelty 5.0 of 10

    Spectra-Scope is a new AutoML framework that trains interpretable machine learning models on spectral data to characterize material properties while enabling users to understand which spectral features drive the predictions.

Reference graph

Works this paper leans on

32 extracted references · 14 canonical work pages · cited by 1 Pith paper

  1. [1]

    Grossman

    Tian Xie and Jeffrey C. Grossman. Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties . Phys. Rev. Lett., 120 0 (14): 0 145301, April 2018. doi:10.1103/PhysRevLett.120.145301. URL https://link.aps.org/doi/10.1103/PhysRevLett.120.145301

    doi:10.1103/physrevlett.120.145301 2018

  2. [2]

    Graph neural networks for materials science and chemistry

    Patrick Reiser, Marlen Neubert, Andr \'e Eberhard, Luca Torresi, Chen Zhou, Chen Shao, Houssam Metni, Clint van Hoesel , Henrik Schopmans, Timo Sommer, and Pascal Friederich. Graph neural networks for materials science and chemistry. Commun Mater, 3 0 (1): 0 1--18, November 2022. ISSN 2662-4443. doi:10.1038/s43246-022-00315-6. URL https://www.nature.com/a...

    doi:10.1038/s43246-022-00315-6 2022

  3. [3]

    Scaling deep learning for materials discovery

    Amil Merchant, Simon Batzner, Samuel S Schoenholz, Muratahan Aykol, Gowoon Cheon, and Ekin Dogus Cubuk. Scaling deep learning for materials discovery. Nature, 624 0 (7990): 0 80--85, 2023

    2023

  4. [4]

    Savitzky, Andrew Minor, Steven Bartholomew Torrisi, Jackson Goedjen, Ching-Chang Chung, Andrew Comstock, and Shijing Sun

    Joseph Harold Montoya, Carolyn Grimley, Muratahan Aykol, Colin Ophus, Hadas Sternlicht, Benjamin H. Savitzky, Andrew Minor, Steven Bartholomew Torrisi, Jackson Goedjen, Ching-Chang Chung, Andrew Comstock, and Shijing Sun. How the AI-assisted discovery and synthesis of a ternary oxide highlights capability gaps in materials science. Chem. Sci., March 2024....

    work page doi:10.1039/d3sc04823c 2024

  5. [5]

    Align before fuse: Vision and language representation learning with momentum distillation

    Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. Align before fuse: Vision and language representation learning with momentum distillation. Advances in neural information processing systems, 34: 0 9694--9705, 2021

    work page 2021

  6. [6]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International conference on machine learning, pages 12888--12900. PMLR, 2022

    work page 2022

  7. [7]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730--19742. PMLR, 2023

    2023

  8. [8]

    Suram, and Weike Ye

    Janghoon Ock, Joseph Montoya, Daniel Schweigert, Linda Hung, Santosh K. Suram, and Weike Ye. UniMat : Unifying Materials Embeddings through Multi-modal Learning , November 2024. URL http://arxiv.org/abs/2411.08664

    arXiv 2024

Show all 32 references
  1. [9]

    Chandran, Jona \"O streicher, Sebastien Bompas, Bashir Kazimi, Stefan Kesselheim, Pascal Friederich, Stefan Sandfeld, and Kevin Maik Jablonka

    Adrian Mirza, Le Yang, Anoop K. Chandran, Jona \"O streicher, Sebastien Bompas, Bashir Kazimi, Stefan Kesselheim, Pascal Friederich, Stefan Sandfeld, and Kevin Maik Jablonka. MatBind : Probing the multimodality of materials science with contrastive learning. In AI for Accelera...

    2025

  2. [10]

    Lu, Thomas Christensen, and Marin Solja c i \'c

    Viggo Moro, Charlotte Loh, Rumen Dangovski, Ali Ghorashi, Andrew Ma, Zhuo Chen, Samuel Kim, Peter Y. Lu, Thomas Christensen, and Marin Solja c i \'c . Multimodal foundation models for material property prediction and discovery. Newton, 1 0 (1), March 2025. ISSN 2950-6360. doi:...

    2025

  3. [11]

    Powder X-Ray Diffraction Pattern Is All You Need for Machine-Learning-Based Symmetry Identification and Property Prediction

    Byung Do Lee, Jin-Woong Lee, Woon Bae Park, Joonseo Park, Min-Young Cho, Satendra Pal Singh, Myoungho Pyo, and Kee-Sun Sohn. Powder X-Ray Diffraction Pattern Is All You Need for Machine-Learning-Based Symmetry Identification and Property Prediction . Advanced Intelligent Syste...

    2022 doi

  4. [12]

    Newnham, Daniele Vivona, Muy Sokseiha, Kiarash Gordiz, Abhishek Aggarwal, Taishan Zhu, Wolfgang G

    Shuo Wang, Sheng Gong, Thorben B \"o ger, Jon A. Newnham, Daniele Vivona, Muy Sokseiha, Kiarash Gordiz, Abhishek Aggarwal, Taishan Zhu, Wolfgang G. Zeier, Jeffrey C. Grossman, and Yang Shao-Horn . Multimodal Machine Learning for Materials Science : Discovery of Novel Li-Ion So...

    2024 doi

  5. [13]

    Kauwe, Ryan J

    Anthony Yu-Tung Wang, Steven K. Kauwe, Ryan J. Murdock, and Taylor D. Sparks. Compositionally restricted attention-based network for materials property predictions. npj Comput Mater, 7 0 (1): 0 1--10, May 2021. ISSN 2057-3960. doi:10.1038/s41524-021-00545-1. URL https://www.na...

    2021 doi

  6. [14]

    Rhys E. A. Goodall and Alpha A. Lee. Predicting materials properties without crystal structure: Deep representation learning from stoichiometry. Nat Commun, 11 0 (1): 0 6280, December 2020. ISSN 2041-1723. doi:10.1038/s41467-020-19964-7. URL https://www.nature.com/articles/s41...

    2020 doi

  7. [15]

    Zagorac, H

    D. Zagorac, H. M \"u ller, S. Ruehl, J. Zagorac, and S. Rehme. Recent developments in the Inorganic Crystal Structure Database : Theoretical crystal structure data and related features. Journal of Applied Crystallography, 52 0 (5): 0 918--925, October 2019. ISSN 1600-5767. doi...

    2019 doi

  8. [16]

    Anubhav Jain, Shyue Ping Ong, Geoffroy Hautier, Wei Chen, William Davidson Richards, Stephen Dacek, Shreyas Cholia, Dan Gunter, David Skinner, Gerbrand Ceder, and Kristin A. Persson. Commentary: The Materials Project : A materials genome approach to accelerating materials inno...

    2013 doi

  9. [17]

    MatFusion : A Multi-Modal Framework Bridging LLMs and Structural Embeddings for Experimental Materials Property Prediction

    Yuwei Wan, Yuqi An, Dongzhan Zhou, Jiahao Dong, Chunyu Kit, Wenjie Zhang, Bram Hoex, Tong Xie, and Yingheng Wang. MatFusion : A Multi-Modal Framework Bridging LLMs and Structural Embeddings for Experimental Materials Property Prediction . In AI for Accelerated Materials Design...

    2025

  10. [18]

    Self-supervised learning of materials concepts from crystal structures via deep neural networks

    Yuta Suzuki, Tatsunori Taniai, Kotaro Saito, Yoshitaka Ushiku, and Kanta Ono. Self-supervised learning of materials concepts from crystal structures via deep neural networks. Mach. Learn.: Sci. Technol., 3 0 (4): 0 045034, December 2022. ISSN 2632-2153. doi:10.1088/2632-2153/a...

    2022 doi

  11. [19]

    Jaakkola

    Tian Xie, Xiang Fu, Octavian-Eugen Ganea, Regina Barzilay, and Tommi S. Jaakkola. Crystal Diffusion Variational Autoencoder for Periodic Material Generation . In International Conference on Learning Representations , October 2021. URL https://openreview.net/forum?id=03RLpj-tc_

    2021

  12. [20]

    Jonathan Schmidt, Noah Hoffmann, Hai-Chen Wang, Pedro Borlido, Pedro J. M. A. Carri c o, Tiago F. T. Cerqueira, Silvana Botti, and Miguel A. L. Marques. Machine- Learning-Assisted Determination of the Global Zero-Temperature Phase Diagram of Materials . Advanced Materials, 35 ...

    2023 doi

  13. [21]

    Persson, Gerbrand Ceder, and Anubhav Jain

    Amalie Trewartha, Nicholas Walker, Haoyan Huo, Sanghoon Lee, Kevin Cruse, John Dagdelen, Alexander Dunn, Kristin A. Persson, Gerbrand Ceder, and Anubhav Jain. Quantifying the advantage of domain-specific pre-training on named entity recognition tasks in materials science. Patt...

    2022

  14. [22]

    Simmtm: A simple pre-training framework for masked time-series modeling

    Jiaxiang Dong, Haixu Wu, Haoran Zhang, Li Zhang, Jianmin Wang, and Mingsheng Long. Simmtm: A simple pre-training framework for masked time-series modeling. Advances in Neural Information Processing Systems, 36: 0 29996--30025, 2023. URL https://proceedings.neurips.cc/paper_fil...

    2023

  15. [23]

    A cookbook of self-supervised learning

    Randall Balestriero, Mark Ibrahim, Vlad Sobal, Ari Morcos, Shashank Shekhar, Tom Goldstein, Florian Bordes, Adrien Bardes, Gregoire Mialon, Yuandong Tian, et al. A cookbook of self-supervised learning. arXiv preprint arXiv:2304.12210, 2023

    2023 arXiv

  16. [24]

    Learning Transferable Visual Models From Natural Language Supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning Transferable Visual Models From Natural Language Supervision . In Proceedings of t...

    2021

  17. [25]

    BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding . In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter ...

    2019

  18. [26]

    BEiT : BERT Pre-Training of Image Transformers

    Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. BEiT : BERT Pre-Training of Image Transformers . In International Conference on Learning Representations , October 2021. URL https://openreview.net/forum?id=p-BhZSz59o4

    2021

  19. [27]

    Chevrier, Kristin A

    Shyue Ping Ong, William Davidson Richards, Anubhav Jain, Geoffroy Hautier, Michael Kocher, Shreyas Cholia, Dan Gunter, Vincent L. Chevrier, Kristin A. Persson, and Gerbrand Ceder. Python Materials Genomics (pymatgen): A robust, open-source python library for materials analysis...

    2013 doi

  20. [28]

    Pytorch: An imperative style, high-performance deep learning library

    A Paszke. Pytorch: An imperative style, high-performance deep learning library. arXiv preprint arXiv:1912.01703, 2019

    1912 arXiv

  21. [29]

    Mixed precision training

    Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, et al. Mixed precision training. arXiv preprint arXiv:1710.03740, 2017

    arXiv

  22. [30]

    Training deep nets with sublinear memory cost

    Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. Training deep nets with sublinear memory cost. arXiv preprint arXiv:1604.06174, 2016

    2016 arXiv

  23. [31]

    Decoupled Weight Decay Regularization

    Ilya Loshchilov and Frank Hutter. Decoupled Weight Decay Regularization . In International Conference on Learning Representations , September 2018. URL https://openreview.net/forum?id=Bkg6RiCqY7

    2018

  24. [32]

    Jonathan Schmidt, Tiago F. T. Cerqueira, Aldo H. Romero, Antoine Loew, Fabian J \"a ger, Hai-Chen Wang, Silvana Botti, and Miguel A. L. Marques. Improving machine-learning models in materials science through large datasets. Materials Today Physics, 48: 0 101560, November 2024....

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.