Tabular foundation models for in-context prediction of molecular properties

Karim K. Ben Hicham , Jan G. Rittig , Martin Grohe , Alexander Mitsos

Authors on Pith no claims yet

Pith reviewed 2026-05-10 08:39 UTC · model grok-4.3

classification 💻 cs.LG physics.chem-ph

keywords molecularfoundationlearningtfmsin-contextmodelspredictionaccurate

0 comments

The pith

Tabular foundation models achieve high accuracy in molecular property prediction through in-context learning, with up to 100% win rates on MoleculeACE tasks when paired with CheMeleon embeddings.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Predicting properties of molecules is important for making new medicines and chemicals. Traditional machine learning needs training a new model for each property, which is slow and needs lots of data. This work instead uses tabular foundation models that look at a few examples right in the input and make predictions without extra training. They test this on standard drug discovery benchmarks and real chemical engineering data. Different ways to describe molecules are tried, including special embeddings from other models and simple lists of chemical features. The results show good accuracy with much less computing power than fine-tuning approaches. The choice of how molecules are represented turns out to matter a lot for how well the tabular models work.

Core claim

combining TFMs with CheMeleon embeddings yields up to 100% win rates on 30 MoleculeACE tasks, while compact RDKit2d and Mordred descriptors provide strong descriptor-based alternatives.

Load-bearing premise

That in-context learning performance with TFMs generalizes reliably to practical low- to medium-data engineering settings and that molecular representation choice drives the gains without hidden task-specific effects or benchmark overfitting.

read the original abstract

Accurate molecular property prediction is central to drug discovery, catalysis, and process design, yet real-world applications are often limited by small datasets. Molecular foundation models provide a promising direction by learning transferable molecular representations; however, they typically involve task-specific fine-tuning, require machine learning expertise, and often fail to outperform classical baselines. Tabular foundation models (TFMs) offer a fundamentally different paradigm: they perform predictions through in-context learning, enabling inference without task-specific training. Here, we evaluate TFMs in the low- to medium-data regime across both standardized pharmaceutical benchmarks and chemical engineering datasets. We evaluate both frozen molecular foundation model representations, as well as classical descriptors and fingerprints. Across the benchmarks, the approach shows excellent predictive performance while reducing computational cost, compared to fine-tuning, with these advantages also transferring to practical engineering data settings. In particular, combining TFMs with CheMeleon embeddings yields up to 100\% win rates on 30 MoleculeACE tasks, while compact RDKit2d and Mordred descriptors provide strong descriptor-based alternatives. Molecular representation emerges as a key determinant in TFM performance, with molecular foundation model embeddings and 2D descriptor sets both providing substantial gains over classic molecular fingerprints on many tasks. These results suggest that in-context learning with TFMs provides a highly accurate and cost-efficient alternative for property prediction in practical applications.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

No free parameters, axioms, or invented entities are described; the work is an empirical comparison of existing models and representations.

pith-pipeline@v0.9.0 · 5551 in / 1033 out tokens · 22645 ms · 2026-05-10T08:39:08.464633+00:00 · methodology

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

TabPFN-3: Technical Report
cs.LG 2026-05 unverdicted novelty 6.0

TabPFN-3 delivers state-of-the-art tabular prediction performance on benchmarks up to 1M rows, is up to 20x faster than prior versions, and introduces test-time scaling that beats non-TabPFN models by hundreds of Elo points.

Reference graph

Works this paper leans on

48 extracted references · 19 canonical work pages · cited by 1 Pith paper · 3 internal anchors

[1]

Yu, M.et al.Deep learning large-scale drug discovery and repurposing.Nature Computational Science4, 600–614 (2024)

2024
[2]

A., Khizbullin, D., Nagaraja, S

Eraqi, B. A., Khizbullin, D., Nagaraja, S. S. & Sarathy, S. M. Molecular prop- erty prediction in the ultra-low data regime.Communications Chemistry8, 201 (2025)

2025
[3]

G., Dahmen, M., Grohe, M., Schwaller, P

Rittig, J. G., Dahmen, M., Grohe, M., Schwaller, P. & Mitsos, A. Molecu- lar machine learning in chemical process design.Current Opinion in Chemical Engineering52, 101239 (2026)

2026
[4]

Altae-Tran, H., Ramsundar, B., Pappu, A. S. & Pande, V. Low Data Drug Discovery with One-Shot Learning.ACS Central Science3, 283–293 (2017)

2017
[5]

V., van Deursen, R

Tetko, I. V., van Deursen, R. & Godin, G. Be aware of overfitting by hyperparameter optimization!Journal of Cheminformatics16, 139 (2024)

2024
[6]

& Czech, W

Praski, M., Adamczyk, J. & Czech, W. Benchmarking Pretrained Molec- ular Embedding Models For Molecular Representation Learning (2025). arXiv:2508.06199

work page arXiv 2025
[7]

Ross, J.et al.Large-scale chemical language representations capture molecular structure and properties.Nature Machine Intelligence4, 1256–1264 (2022)

2022
[8]

& Farimani, A

Wang, Y., Wang, J., Cao, Z. & Farimani, A. B. Molecular Contrastive Learning of Representations via Graph Neural Networks.Nature Machine Intelligence4, 279–287 (2022)

2022
[9]

Descriptor-Based

Burns, J. W.et al.Deep Learning Foundation Models from Classical Molecular Descriptors (2026). arXiv:2506.15792

work page arXiv 2026
[10]

Fine-tuning can distort pretrained features and underperform out-of-distribution.arXiv preprint arXiv:2202.10054,

Kumar, A., Raghunathan, A., Jones, R., Ma, T. & Liang, P. Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution (2022). arXiv:2202.10054

work page arXiv 2022
[11]

& Varoquaux, G

Grinsztajn, L., Oyallon, E. & Varoquaux, G. Why do tree-based models still out- perform deep learning on typical tabular data?Advances in Neural Information Processing Systems35, 507–520 (2022)

2022
[12]

& Grisoni, F

van Tilborg, D., Alenicheva, A. & Grisoni, F. Exposing the Limitations of Molec- ular Machine Learning with Activity Cliffs.Journal of Chemical Information and Modeling62, 5938–5951 (2022)

2022
[13]

Hollmann, N.et al.Accurate predictions on small data with a tabular foundation model.Nature637, 319–326 (2025). 17

2025
[14]

TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second

Hollmann, N., M¨ uller, S., Eggensperger, K. & Hutter, F. TabPFN: A Trans- former That Solves Small Tabular Classification Problems in a Second (2023). arXiv:2207.01848

work page internal anchor Pith review arXiv 2023
[15]

Tabiclv2: A better, faster, scalable, and open tabular foundation model, 2026

Qu, J., Holzm¨ uller, D., Varoquaux, G. & Morvan, M. L. TabICLv2: A better, faster, scalable, and open tabular foundation model (2026). arXiv:2602.11139

work page arXiv 2026
[16]

TabICL: A tabular foundation model for in-context learning on large data.arXiv preprint arXiv:2502.05564, 2025

Qu, J., Holzm¨ uller, D., Varoquaux, G. & Morvan, M. L. TabICL: A Tabular Foun- dation Model for In-Context Learning on Large Data (2025). arXiv:2502.05564

work page arXiv 2025
[17]

Tabarena: A living benchmark for machine learning on tabular data.arXiv preprint arXiv:2506.16791, 2025

Erickson, N.et al.TabArena: A Living Benchmark for Machine Learning on Tabular Data (2025). arXiv:2506.16791

work page arXiv 2025
[18]

Grinsztajn, L.et al.Tabpfn-2.5: Advancing the state of the art in tabular foundation models.arXiv preprint arXiv:2511.08667(2025)

work page internal anchor Pith review arXiv 2025
[19]

Chen, W.et al.TabPFN Opens New Avenues for Small-Data Tabular Learning in Drug Discovery.Journal of Chemical Information and Modelingacs.jcim.5c02823 (2026)

2026
[20]

URL https://doi.org/10.5281/zenodo.18797641

Landrum, G., Tosco, P., Kelley, B.et al.Rdkit: Open-source cheminformatics (2026). URL https://doi.org/10.5281/zenodo.18797641

work page doi:10.5281/zenodo.18797641 2026
[21]

& Takagi, T

Moriwaki, H., Tian, Y.-S., Kawashita, N. & Takagi, T. Mordred: A molecular descriptor calculator.Journal of Cheminformatics10, 4 (2018)

2018
[22]

Soares, E.et al.An open-source family of large encoder-decoder foundation models for chemistry.Communications Chemistry8, 193 (2025)

2025
[23]

& Klambauer, G

Seidl, P., Vall, A., Hochreiter, S. & Klambauer, G. Enhancing Activity Predic- tion Models in Drug Discovery with the Ability to Understand Human Language (2023). arXiv:2303.03363

work page arXiv 2023
[24]

Heid, E.et al.Chemprop: a machine learning package for chemical property prediction.Journal of chemical information and modeling64, 9–17 (2023)

2023
[25]

Daniel E Koshland Jr

Kl¨ aser, K.et al.MiniMol: A Parameter-Efficient Foundation Model for Molecular Learning (2024). arXiv:2404.14986

work page arXiv 2024
[26]

P., Wang, W

Sheridan, R. P., Wang, W. M., Liaw, A., Ma, J. & Gifford, E. M. Extreme Gra- dient Boosting as a Method for Quantitative Structure–Activity Relationships. Journal of Chemical Information and Modeling56, 2353–2360 (2016)

2016
[27]

Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A. V. & Gulin, A. Catboost: unbiased boosting with categorical features.Advances in neural information processing systems31(2018). 18

2018
[28]

R.et al.Practically Significant Method Comparison Protocols for Machine Learning in Small Molecule Drug Discovery.Journal of Chemical Information and Modeling65, 9398–9411 (2025)

Ash, J. R.et al.Practically Significant Method Comparison Protocols for Machine Learning in Small Molecule Drug Discovery.Journal of Chemical Information and Modeling65, 9398–9411 (2025)

2025
[29]

Huang, T

Huang, K.et al.Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development (2021). arXiv:2102.09548

work page arXiv 2021
[30]

M.et al.Graph neural networks for prediction of fuel ignition quality.Energy & fuels34, 11395–11407 (2020)

Schweidtmann, A. M.et al.Graph neural networks for prediction of fuel ignition quality.Energy & fuels34, 11395–11407 (2020)

2020
[31]

& Rittig, J

bin Javaid, M., Gervens, T., Mitsos, A., Grohe, M. & Rittig, J. G. Exploring data augmentation: Multi-task methods for molecular property prediction.Computers & Chemical Engineering201, 109253 (2025)

2025
[32]

& Ramprasad, R

Kuenneth, C. & Ramprasad, R. polybert: a chemical language model to enable fully machine-driven ultrafast polymer informatics.Nature communications14, 4099 (2023)

2023
[33]

& Li, Y.-P

Liao, H.-C., Lin, Y.-H., Peng, C.-H. & Li, Y.-P. Directed message passing neural networks for accurate prediction of polymer–solvent interaction parameters.ACS Engineering Au5, 530–539 (2025)

2025
[34]

Superior molecular representations from intermediate encoder layers

Pinto, L. Superior molecular representations from intermediate encoder layers. arXiv preprint arXiv:2506.06443(2025)

work page arXiv 2025
[35]

Wognum, C.et al.A call for an industry-led initiative to critically assess machine learning for real-world drug discovery.Nature Machine Intelligence6, 1120–1121 (2024)

2024
[36]

Zhou, J., Yang, Y., Mroz, A. M. & Jelfs, K. E. Polycl: contrastive learning for polymer representation learning via explicit and implicit augmentations.Digital Discovery4, 149–160 (2025)

2025
[37]

& Barati Farimani, A

Xu, C., Wang, Y. & Barati Farimani, A. Transpolymer: a transformer-based language model for polymer property predictions.npj Computational Materials 9, 64 (2023)

2023
[38]

Erickson, N.et al.Autogluon-tabular: Robust and accurate automl for structured data.arXiv preprint arXiv:2003.06505(2020)

work page internal anchor Pith review arXiv 2003
[39]

Sinodinos, D.et al.Multitask-informed prior for in-context learning on tabular data: Application to steel property prediction.arXiv preprint arXiv:2603.22738 (2026)

work page arXiv 2026
[40]

Sinodinos, D., Wei, J. Y. & Armanfard, N. Multitab: A scalable foundation for multitask learning on tabular data.Proceedings of the AAAI Conference on Artificial Intelligence(2026). 19

2026
[41]

T.-Y., Picard, C

Yu, R. T.-Y., Picard, C. & Ahmed, F. GIT-BO: High-Dimensional Bayesian Optimization with Tabular Foundation Models (2026). arXiv:2505.20685

work page arXiv 2026
[42]

Transformers can do bayesian inference, 2024

M¨ uller, S., Hollmann, N., Arango, S. P., Grabocka, J. & Hutter, F. Transformers Can Do Bayesian Inference (2024). arXiv:2112.10510

work page arXiv 2024
[43]

Garg, A.et al.Real-tabpfn: Improving tabular foundation models via continued pre-training with real-world data.arXiv preprint arXiv:2507.03971(2025)

work page arXiv 2025
[44]

TabPFN (2026)

PriorLabs. TabPFN (2026). URL https://github.com/PriorLabs/TabPFN. GitHub repository, accessed 2026-04-09

2026
[45]

Mark, J. E. & Mark, J. E.Physical properties of polymers handbookVol. 1076 (Springer, 2007)

2007
[46]

K., Rittig, J

Ben Hicham, K. K., Rittig, J. G., Grohe, M. & Mitsos, A. Tabular foundation models for in-context prediction of molecular properties (2026). URL https:// doi.org/10.5281/zenodo.19605631

work page doi:10.5281/zenodo.19605631 2026
[47]

Medina, E. I. S., Linke, S., Stoll, M. & Sundmacher, K. Gibbs–helmholtz graph neural network: capturing the temperature dependency of activity coefficients at infinite dilution.Digital Discovery2, 781–798 (2023)

2023
[48]

catalaix

Brouwer, T., Kersten, S. R., Bargeman, G. & Schuur, B. Trends in solvent impact on infinite dilution activity coefficients of solutes reviewed and visualized using an algorithm to support selection of solvents for greener fluid separations.Separation and Purification Technology272, 118727 (2021). Author Contributions Karim K. Ben Hicham: Conceptualization...

2021