TIME: TabPFN-Integrated Multimodal Engine for Robust Tabular-Image Learning

Jiaqi Luo; Shixin Xu; Yuan Yuan

REVIEW 3 major objections 5 minor 2 cited by

TIME: TabPFN-Integrated Multimodal Engine for Robust Tabular-Image Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a frozen TabPFN tabular encoder, fused with image features, beats task-trained tabular encoders in multimodal learning and that raw incomplete tables beat median-imputed ones.

desk verdict Useful recipe for injecting frozen TabPFN into multimodal fusion, but the training-set embedding protocol leaks labels and the evaluation misses the most relevant baseline. read the letter →

arxiv 2506.00813 v1 pith:EALDPAJD submitted 2025-06-01 cs.CV cs.LG

Jiaqi Luo , Yuan Yuan , Shixin Xu This is my paper · ORCID

classification cs.CVcs.LG

keywords multimodallearningtabular-imagefusionTabPFNmissingvaluestabularfoundationmodelmedicalimagingrobustnesspretrainedencoder

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that tabular-image multimodal learning can be made more robust by borrowing a pretrained tabular foundation model, TabPFN, as a frozen encoder. It argues that this approach removes the need for task-specific tabular training and avoids median imputation, which can distort data distributions. The proposed TIME framework fuses TabPFN embeddings with pretrained image features and, in experiments on five datasets, outperforms MLP and NCART baselines under both complete and missing tabular inputs. A sympathetic reader would care because medical datasets are small, heterogeneous, and full of missing values, which is exactly the regime where pretrained tabular representations could help.

What carries the argument

The load-bearing object is the frozen TabPFN encoder $f_{\mathrm{TabPFN}}(X, D_{\mathrm{train}})$: a transformer-based tabular foundation model pretrained on synthetic tasks with in-context learning, which maps each tabular row to a 192-dimensional embedding while conditioning on the whole training set and natively tolerating missing entries. This embedding is fused with a ResNet-50 image embedding by concatenation, element-wise sum, element-wise maximum, or a DAFT transform, and the fused vector feeds a linear classifier or regressor. Keeping TabPFN frozen is what preserves its zero-shot generalization and missing-value behavior; only the projection layers and, optionally, the image encoder are trained.

What would settle it

A direct test is to take a medical dataset where missingness is known to be informative, such as lab results absent more often for severe cases, and compare TIME trained on raw incomplete tables against the same model trained on median-imputed tables. If the imputed version matches or beats the raw version, or if a strong imputation-based baseline does, then the claim that TabPFN's native missing-value handling is the source of robustness fails.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that a pretrained tabular foundation model can serve as a drop-in, frozen tabular encoder for multimodal learning: TIME, built by fusing TabPFN's 192-dimensional row embeddings with ResNet-50 image features, beats both a task-trained MLP and the NCART tree-based encoder on five datasets, and does so by larger margins on medical imaging data. The paper further reports that training TIME on raw, incomplete tabular data yields higher accuracy than training on median-imputed data across all three medical datasets, and that performance degrades only mildly when up to half the tabular entries are randomly masked. The claimed mechanism is that TabPFN's native missing-value handling and pretrained in-context representations transfer to the multimodal setting, so imputation, which can distort distributions, becomes unnecessary.

Load-bearing premise

The robustness conclusion relies on random missingness being a fair stand-in for the real missingness in medical tables, yet the paper never examines why values are missing in its medical datasets; if missingness carries information about the outcome, TabPFN's advantage over imputation may not transfer.

Editorial extensions

If this is right

Adding tabular information through TIME improves over a vision-only ResNet on all five datasets, so the paper's recipe benefits natural images, paintings, and medical images alike.
TIME's gains over MLP and NCART are largest on the three medical datasets, suggesting pretrained tabular embeddings matter most where data are small and heterogeneous.
Because raw incomplete inputs outperform median-imputed inputs in every medical dataset and fusion configuration, the paper implies that imputation is not just unnecessary but can be harmful in this pipeline.
Under random missingness up to 50%, TIME's accuracy stays near its complete-data level, with concatenation and max fusion the most resilient strategies.
The best fusion strategy depends on encoder and setting, with concatenation and DAFT broadly strong and sum fusion excelling with TabPFN under full fine-tuning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension is to swap TabPFN for any future tabular foundation model with native missing-value support; the paper's architecture does not depend on TabPFN-specific details beyond the embedding output, so its core claim would be testable as better tabular pretraining arrives.
Because TabPFN's embedding for one row depends on the entire training set via in-context conditioning, TIME's advantage may shrink as datasets approach TabPFN's stated limits of 10,000 samples, 500 features, and 10 classes; the paper acknowledges this constraint but does not test it.
The sensitivity analysis masks only the Adoption dataset, so the paper's inference about medical missingness would be stronger if random masking were applied to the medical datasets and compared with missingness-correlated masking, which would test whether native missing-value handling survives informative missingness.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The core idea is simple and worth testing: use frozen TabPFN as a tabular encoder in an image-tabular pipeline, and let its native missing-value handling replace imputation. The paper runs a serious set of experiments, five seeds, standard deviations, four fusion strategies, on natural and medical datasets, and the results mostly support the claim that TabPFN embeddings help. That is a genuinely new application-level contribution, even if the fusion modules themselves are off-the-shelf.

The soft spots are real, though. The load-bearing one is Eq. (3). You compute embeddings as f(X*, Dtrain), and the text explicitly says X* may be a training row. TabPFN is an in-context predictor, so a training row's embedding can attend to its own label inside Dtrain. At test time the query label is absent. That means the linear head is trained on features that have a different distribution from the features it sees at test time, while the MLP and NCART baselines never see labels in their tabular features. The direction of the bias is not clear—it could plausibly hurt test accuracy by teaching the head to rely on a shortcut—but it certainly breaks the clean comparison. The authors need to compute training embeddings leave-one-out with respect to Dtrain to isolate what TabPFN's pretrained representation actually contributes.

Also missing is TIP, which is cited in the related work and is the most relevant baseline for incomplete tabular-image learning. Leaving it out substantially weakens the claim of robustness against prior art. The sensitivity analysis is MCAR masking on one natural-image dataset; it does not address the informative missingness that medical data often has, and the paper never analyzes the actual missingness patterns in its three medical datasets. There is also an internal overclaim: Section 4.2 says TIME is superior in best and average scores, but Table 1 shows NCART-Max beating TIME-DAFT on Painting in the frozen setting. Minor, but the phrasing needs adjusting.

Credit where due: the paper is honest about TabPFN's size limits, reports code will be released only upon acceptance, and the empirical effort is well above the subfield average. The limitation section is candid.

This deserves a serious referee, but not acceptance as is. The leakage issue needs a leave-one-out protocol, TIP needs to be run, and the missingness analysis needs to face real missingness mechanisms. I would send it back for major revision.

Referee Report

3 major / 5 minor

Summary. The paper proposes TIME, a multimodal framework that uses a frozen TabPFN as a tabular encoder and combines its embeddings with pretrained ResNet image features through several fusion strategies (concatenation, sum, max, DAFT). The method is evaluated on five datasets (Adoption, Breast Cancer, Covid-19, Skin Cancer, Painting) under fully tuned and frozen image-encoder settings, and on three medical datasets with real missing values. The main empirical claims are that TIME outperforms MLP and NCART baselines on complete data, and that training directly on raw incomplete data gives better results than median imputation. The paper also reports a random-masking sensitivity analysis on the Adoption dataset.

Significance. If the results hold, TIME would be a useful demonstration that a pretrained tabular foundation model can serve as a drop-in tabular encoder for multimodal tabular-image learning, while also providing native handling of missing values. The paper has concrete strengths: experiments are repeated over five seeds with standard deviations, multiple fusion strategies and both frozen/fine-tuned settings are considered, the raw-vs-imputed comparison directly tests a practical claim, and the code is promised for release upon acceptance. However, a potentially load-bearing target-leakage issue in the embedding protocol, the absence of the most relevant missingness-aware baseline (TIP), and the narrow sensitivity analysis make the current evidence conditional rather than conclusive.

major comments (3)

[§3.1–3.2, Eq. (3)] The manuscript defines the tabular embedding as E = f(X*, Dtrain) and explicitly states that X* may belong to training, validation, or test sets. When X* is a training row, its own label y_i is present in Dtrain. Since TabPFN is a transformer trained for in-context prediction, the embedding for that row can attend to its own label in the support set and thereby encode the target value. At test time the query label is absent, so the linear head in Eqs. (5)–(6) is trained on feature vectors whose distribution differs from the test distribution, while the MLP and NCART baselines never expose the label in their tabular features. This covariate shift can inflate TIME's apparent advantage in Table 1 even if TabPFN's representation is not genuinely better. The paper does not state that training embeddings are computed leave-one-out; please specify the exact protocol (e.g., leave-one-out support construction or a held-out support set) and, if leakage is present, rerun the comparison.
[§2.2 and §4.3, Table 2] TIP [12] is cited in the related work as addressing missingness in downstream tabular data in a tabular-image setting, yet it is not included as a baseline in Table 2, and Section 2.2 claims to be 'the first to address tabular-image multimodal learning with missing values in a supervised setting.' Because the robustness evaluation in Table 2 compares TIME on raw data only against TIME on median-imputed data, the broader conclusion that TIME 'consistently outperforms competitive baselines' under missingness is not supported by the experiments. Please compare against TIP (or, if its code/data are unavailable, state this explicitly and justify the scope) and reconcile the novelty claim with the cited work.
[§4.4 and Table A.3] The sensitivity analysis randomly masks tabular features on the Adoption dataset only, and the manuscript does not characterize the missingness mechanisms of the three medical datasets beyond the sample counts in Table A.3. The claim that TabPFN's native missingness handling is robust therefore rests on the untested assumption that random masking resembles real clinical missingness. Please report per-feature missing rates and, if feasible, evaluate under informative (e.g., outcome-dependent) missingness mechanisms or otherwise restrict the claim.

minor comments (5)

[§4.2, Table 1] The sentence 'TIME models outperform both NCART and MLP baselines, achieving superior results in terms of both individual best scores and average performance' is too strong: on Painting in the frozen setting, NCART-Max (16.69) beats the best TIME variant (TIME-DAFT, 17.01). Please qualify the claim to reflect the few settings where a baseline wins.
[§4.3 and Figures 4–5] The labels F- and R- are used in the figures for imputed versus raw data, but the text reuses them in a way that is confusing (e.g., 'F-Cat under the frozen setting' refers to a value that in Figure 5 is actually the raw R-Cat result, 73.81%). Consider renaming the conditions (e.g., Imputed vs Raw) and aligning the text with the figure labels.
[Table 2 caption] The caption says 'the bold denotes the best average performance' but the bold formatting appears to mark the best result for each dataset; please clarify the formatting convention.
[§4.1] The MLP and NCART baselines are trained only after median imputation; please state explicitly whether the same imputation was applied at test time and whether any hyperparameter tuning was performed for these baselines beyond the shared training schedule.
[Figure 6] The caption refers to 'the three dotted lines' for baselines, but the baseline labels in the figure legend are not clearly visible in the printed version; add explicit line labels or a legend key so the comparison is readable.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the central claims are empirical comparisons against external baselines, and the only self-citation (NCART) is a comparator rather than a load-bearing premise.

full rationale

The paper makes no first-principles derivation whose output is equivalent to its input. Its central claims are (i) TIME outperforms MLP/NCART/ResNet baselines (Section 4.2, Table 1) and (ii) training on raw incomplete data beats median-imputed data (Section 4.3, Table 2). Both are direct empirical tests against external baselines or against an imputation baseline. TabPFN is an externally published frozen foundation model (Hollmann et al., Nature 2025), not a component built and then retroactively justified by this paper. The only self-citation is NCART [20] (Luo & Xu), used as a comparative tabular encoder; it is not cited as evidence for the method's correctness, so it is not load-bearing. The raw-versus-imputed experiment directly tests TabPFN's claimed native missing-value handling, so that conclusion is not assumed. Eq. (3)'s convention that training-set inputs are embedded with Dtrain containing their own labels is a potential feature-level leakage caveat for interpreting the representation's contribution, but the reported test metrics are computed on held-out labels and the claim is empirical, so this is a methodological validity concern rather than a circularity of the derivation. The limitations in Section 5 are honest constraints (TabPFN scale limits, simple fusion) and do not conceal a circular step. The random-masking sensitivity analysis in Section 4.4 may not reflect real missingness mechanisms, but this is an external-validity concern, not a circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters or invented entities; the method relies on domain assumptions about TabPFN's transferability and the representativeness of the datasets and masking procedure.

assumptions (3)

domain assumption The internal encoder output of TabPFN, when conditioned on Dtrain, provides a stable and informative feature vector for a separate linear classifier.
The method assumes the internal encoder output of TabPFN, trained for in-context prediction, serves as a useful frozen feature extractor in a multimodal fusion (Section 3.2).
ad hoc to paper Random feature masking in the sensitivity analysis approximates real-world missing-data mechanisms.
Section 4.4 masks values at random on the Adoption dataset; the paper generalizes the robustness finding to medical datasets without analyzing their missingness mechanisms.
domain assumption The five chosen datasets are representative of tabular-image learning tasks.
The paper claims consistent improvements across diverse domains from these five datasets (Appendix A).

how reviews work

0 comments

Cite this review

Pith. "Pith review of TIME: TabPFN-Integrated Multimodal Engine for Robust Tabular-Image Learning." pith.science (2026). https://pith.science/paper/EALDPAJD

@misc{pith2026250600813,
  author       = {Pith},
  title        = {Pith review of: TIME: TabPFN-Integrated Multimodal Engine for Robust Tabular-Image Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EALDPAJD}},
  note         = {Machine review of arXiv:2506.00813}
}

read the original abstract

Tabular-image multimodal learning, which integrates structured tabular data with imaging data, holds great promise for a variety of tasks, especially in medical applications. Yet, two key challenges remain: (1) the lack of a standardized, pretrained representation for tabular data, as is commonly available in vision and language domains; and (2) the difficulty of handling missing values in the tabular modality, which are common in real-world medical datasets. To address these issues, we propose the TabPFN-Integrated Multimodal Engine (TIME), a novel multimodal framework that builds on the recently introduced tabular foundation model, TabPFN. TIME leverages TabPFN as a frozen tabular encoder to generate robust, strong embeddings that are naturally resilient to missing data, and combines them with image features from pretrained vision backbones. We explore a range of fusion strategies and tabular encoders, and evaluate our approach on both natural and medical datasets. Extensive experiments demonstrate that TIME consistently outperforms competitive baselines across both complete and incomplete tabular inputs, underscoring its practical value in real-world multimodal learning scenarios.

Figures

Figures reproduced from arXiv: 2506.00813 by the authors.

**Figure 2.** Rankings under the fully tuned setting on five datasets. [PITH_FULL_IMAGE:figures/full_fig_p017_2.png] view at source ↗

**Figure 3.** Rankings under the frozen setting on five datasets. [PITH_FULL_IMAGE:figures/full_fig_p018_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Comparison of the fully tuned setting on three medical datasets. Dark-colored [PITH_FULL_IMAGE:figures/full_fig_p021_4.png]

**Figure 5.** Figure 5: Comparison of the frozen setting on three medical datasets. Dark-colored bars [PITH_FULL_IMAGE:figures/full_fig_p022_5.png]

**Figure 6.** Figure 6: Results of TIME under different masking ratios on the Adoption dataset, eval [PITH_FULL_IMAGE:figures/full_fig_p023_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Brain Vascular Age Prediction Using Cerebral Blood Flow Velocity and Machine Learning Algorithms
cs.AI 2026-05 unverdicted novelty 5.0 of 10

TCD-derived MOCAIP and HRV features predict chronological age in healthy adults (MAE ~3.7 y), and diseased cohorts show larger errors that the authors interpret as accelerated cerebrovascular aging.
Adaptation and Fine-tuning with TabPFN for Travelling Salesman Problem
cs.LG 2025-11 conditional novelty 5.0 of 10

TabPFN-v2, fine-tuned on one 500-node TSP sample in about two minutes, constructs TSP tours reaching 2-5% of Concorde optimality after 2-opt post-processing, across instance sizes 50 to 1000.

Reference graph

Works this paper leans on

35 extracted references · 26 canonical work pages · cited by 2 Pith papers

[12]

S. Du, S. Zheng, Y. Wang, W. Bai, D. P. O’Regan, C. Qin, Tip: Tabular- image pre-training for multimodal classification with incomplete data, in: European Conference on Computer Vision, Springer, 2024, pp. 478– 496

work page 2024
[1]

Baltruˇ saitis, C

T. Baltruˇ saitis, C. Ahuja, L.-P. Morency, Multimodal machine learning: A survey and taxonomy, IEEE transactions on pattern analysis and machine intelligence 41 (2) (2018) 423–443. 27 Table A.3: Dataset details. #Samples indicates the total number of samples in the dataset; #Features refers to the number of tabular features; #Classes denotes the number o...

work page 2018
[2]

J. N. Acosta, G. J. Falcone, P. Rajpurkar, E. J. Topol, Multimodal biomedical ai, Nature medicine 28 (9) (2022) 1773–1784

work page 2022
[3]

Huang, A

S.-C. Huang, A. Pareek, S. Seyyedi, I. Banerjee, M. P. Lungren, Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines, NPJ digital medicine 3 (1) (2020) 136

work page 2020
[4]

Duanmu, P

H. Duanmu, P. B. Huang, S. Brahmavar, S. Lin, T. Ren, J. Kong, F. Wang, T. Q. Duong, Prediction of pathological complete response to neoadjuvant chemotherapy in breast cancer using deep learning with integrative imaging, molecular and demographic data, in: Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conferenc...

work page 2020
[5]

Borsos, C

B. Borsos, C. G. Allaart, A. van Halteren, Predicting stroke outcome: a 28 case for multimodal deep learning methods with tabular and ct perfusion data, Artificial Intelligence in Medicine 147 (2024) 102719

work page 2024
[6]

L. A. Vale-Silva, K. Rohr, Long-term cancer survival prediction using multimodal deep learning, Scientific Reports 11 (1) (2021) 13505

2021
[7]

K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778

2016

Show all 35 references

[8]

Dosovitskiy, L

A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al., An image is worth 16x16 words: Transformers for image recognition at scale, arXiv preprint arXiv:2010.11929 (2020)

2020 arXiv
[9]

Devlin, M.-W

J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, in: Pro- ceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies,...

2019
[10]

Jiang, H.-J

J.-P. Jiang, H.-J. Ye, L. Wang, Y. Yang, Y. Jiang, D.-C. Zhan, Tabular insights, visual impacts: transferring expertise from tables to images, in: Forty-first International Conference on Machine Learning, 2024

2024
[11]

Holste, S

G. Holste, S. C. Partridge, H. Rahbar, D. Biswas, C. I. Lee, A. M. Alessio, End-to-end learning of fused image and non-image features for 29 improved breast cancer classification from mri, in: Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 3...

2021
[13]

Hager, M

P. Hager, M. J. Menten, D. Rueckert, Best of both worlds: Multimodal contrastive learning with tabular and imaging data, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, 2023, pp. 23924–23935

2023
[14]

W. Huang, Multimodal contrastive learning and tabular attention for automated alzheimer’s disease prediction, in: Proceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 2473– 2482

2023
[15]

T. Chen, C. Guestrin, Xgboost: A scalable tree boosting system, in: Proceedings of the 22nd acm sigkdd international conference on knowl- edge discovery and data mining, 2016, pp. 785–794

2016
[16]

G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, T.-Y. Liu, Lightgbm: A highly efficient gradient boosting decision tree, Advances in neural information processing systems 30 (2017). 30

2017
[17]

Prokhorenkova, G

L. Prokhorenkova, G. Gusev, A. Vorobev, A. V. Dorogush, A. Gulin, Catboost: unbiased boosting with categorical features, Advances in neu- ral information processing systems 31 (2018)

2018
[18]

Z. Wang, J. Sun, Transtab: Learning transferable tabular transformers across tables, Advances in Neural Information Processing Systems 35 (2022) 2902–2915

2022
[19]

Borisov, T

V. Borisov, T. Leemann, K. Seßler, J. Haug, M. Pawelczyk, G. Kasneci, Deep neural networks and tabular data: A survey, IEEE Transactions on Neural Networks and Learning Systems (2022)

2022
[20]

J. Luo, S. Xu, Ncart: Neural classification and regression tree for tabular data, Pattern Recognition 154 (2024) 110578

2024
[21]

J. Yoon, Y. Zhang, J. Jordon, M. van der Schaar, Vime: Extending the success of self-and semi-supervised learning to tabular domain, Advances in Neural Information Processing Systems 33 (2020) 11033–11043

2020
[22]

Bahri, H

D. Bahri, H. Jiang, Y. Tay, D. Metzler, Scarf: Self-supervised con- trastive learning using random feature corruption, arXiv preprint arXiv:2106.15147 (2021)

2021 arXiv
[23]

L. Xu, M. Skoularidou, A. Cuesta-Infante, K. Veeramachaneni, Model- ing tabular data using conditional gan, Advances in neural information processing systems 32 (2019). 31

2019
[24]

Huang, A

X. Huang, A. Khetan, M. Cvitkovic, Z. Karnin, Tabtransformer: Tabular data modeling using contextual embeddings, arXiv preprint arXiv:2012.06678 (2020)

2020 arXiv
[25]

Somepalli, M

G. Somepalli, M. Goldblum, A. Schwarzschild, C. B. Bruss, T. Gold- stein, Saint: Improved neural networks for tabular data via row at- tention and contrastive pre-training, arXiv preprint arXiv:2106.01342 (2021)

2021 arXiv
[26]

Hollmann, S

N. Hollmann, S. M¨ uller, L. Purucker, A. Krishnakumar, M. K¨ orfer, S. B. Hoo, R. T. Schirrmeister, F. Hutter, Accurate predictions on small data with a tabular foundation model, Nature 637 (8045) (2025) 319–326

2025
[27]

Y. Lei, Z. Li, Y. Shen, J. Zhang, H. Shan, Clip-lung: Textual knowledge- guided lung nodule malignancy prediction, in: International Confer- ence on Medical Image Computing and Computer-Assisted Intervention, Springer, 2023, pp. 403–412

2023
[28]

L. Sun, M. Zhang, Y. Lu, W. Zhu, Y. Yi, F. Yan, Nodule-clip: Lung nodule classification based on multi-modal contrastive learning, Com- puters in Biology and Medicine 175 (2024) 108505

2024
[29]

Spasov, L

S. Spasov, L. Passamonti, A. Duggento, P. Lio, N. Toschi, A. D. N. Initiative, et al., A parameter-efficient deep learning approach to pre- dict conversion from mild cognitive impairment to alzheimer’s disease, Neuroimage 189 (2019) 276–287. 32

2019
[30]

Y. Liu, Y. Yu, J. Ouyang, B. Jiang, G. Yang, S. Ostmeier, M. Winter- mark, P. Michel, D. S. Liebeskind, M. G. Lansberg, et al., Functional outcome prediction in acute ischemic stroke using a fused imaging and clinical deep learning model, Stroke 54 (9) (2023) 2316–2327

2023
[31]

Zheng, Z

H. Zheng, Z. Lin, Q. Zhou, X. Peng, J. Xiao, C. Zu, Z. Jiao, Y. Wang, Multi-transsp: Multimodal transformer for survival prediction of na- sopharyngeal carcinoma patients, in: International Conference on Med- ical Image Computing and Computer-Assisted Intervention, Springer, 2...

2022
[32]

C. Xue, S. S. Kowshik, D. Lteif, S. Puducheri, V. H. Jasodanand, O. T. Zhou, A. S. Walia, O. B. Guney, J. D. Zhang, S. T. Pham, et al., Ai- based differential diagnosis of dementia etiologies on multimodal data, Nature Medicine 30 (10) (2024) 2977–2989

2024
[33]

T. N. Wolf, S. P¨ olsterl, C. Wachinger, A. D. N. Initiative, et al., Daft: A universal module to interweave tabular data and 3d images in cnns, NeuroImage 260 (2022) 119505

2022
[34]

N. Tang, R. Zhang, Z. Wei, X. Chen, G. Li, Q. Song, D. Yi, Y. Wu, Improving the performance of lung nodule classification by fusing struc- tured and unstructured data, Information Fusion 88 (2022) 161–174

2022
[35]

Hollmann, S

N. Hollmann, S. M¨ uller, K. Eggensperger, F. Hutter, Tabpfn: A trans- 33 former that solves small tabular classification problems in a second, arXiv preprint arXiv:2207.01848 (2022). 34

2022 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [12]

S. Du, S. Zheng, Y. Wang, W. Bai, D. P. O’Regan, C. Qin, Tip: Tabular- image pre-training for multimodal classification with incomplete data, in: European Conference on Computer Vision, Springer, 2024, pp. 478– 496

work page 2024

[2] [1]

Baltruˇ saitis, C

T. Baltruˇ saitis, C. Ahuja, L.-P. Morency, Multimodal machine learning: A survey and taxonomy, IEEE transactions on pattern analysis and machine intelligence 41 (2) (2018) 423–443. 27 Table A.3: Dataset details. #Samples indicates the total number of samples in the dataset; #Features refers to the number of tabular features; #Classes denotes the number o...

work page 2018

[3] [2]

J. N. Acosta, G. J. Falcone, P. Rajpurkar, E. J. Topol, Multimodal biomedical ai, Nature medicine 28 (9) (2022) 1773–1784

work page 2022

[4] [3]

Huang, A

S.-C. Huang, A. Pareek, S. Seyyedi, I. Banerjee, M. P. Lungren, Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines, NPJ digital medicine 3 (1) (2020) 136

work page 2020

[5] [4]

Duanmu, P

H. Duanmu, P. B. Huang, S. Brahmavar, S. Lin, T. Ren, J. Kong, F. Wang, T. Q. Duong, Prediction of pathological complete response to neoadjuvant chemotherapy in breast cancer using deep learning with integrative imaging, molecular and demographic data, in: Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conferenc...

work page 2020

[6] [5]

Borsos, C

B. Borsos, C. G. Allaart, A. van Halteren, Predicting stroke outcome: a 28 case for multimodal deep learning methods with tabular and ct perfusion data, Artificial Intelligence in Medicine 147 (2024) 102719

work page 2024

[7] [6]

L. A. Vale-Silva, K. Rohr, Long-term cancer survival prediction using multimodal deep learning, Scientific Reports 11 (1) (2021) 13505

2021

[8] [7]

K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778

2016