Pith. sign in

REVIEW 3 major objections 6 minor 89 references

FoMoH: A clinically meaningful foundation model evaluation for structured electronic health records

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FoMoH benchmarks six EHR foundation models on 14 clinical tasks and finds their frozen embeddings do not beat supervised baselines where labels are rare.

desk verdict A solid, genuinely useful EHR foundation-model benchmark whose central negative finding is real but specifically about linear probing, not about the models under fine-tuning. read the letter →

arxiv 2505.16941 v3 pith:4BOOTXZC submitted 2025-05-22 cs.LG cs.AI

Chao Pang , Vincent Jeanselme , Young Sang Choi , Xinzhuo Jiang , Zilin Jing , Aparajita Kashyap , Yuta Kobayashi , Yanwei Li
show 3 more authors
Florent Pollet Karthik Natarajan Shalmali Joshi
This is my paper · ORCID
classification cs.LGcs.AI
keywords foundationmodelselectronichealthrecordslinearprobingtemporalmodelingclinicalbenchmarkcalibrationfairnesshealthcaremachinelearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper builds a benchmark, FoMoH, to test whether structured electronic health record (EHR) foundation models actually deliver clinically useful representations. It evaluates six models, pre-trained from scratch on about 5.3 million patients' records, on 14 tasks spanning mortality, readmission, long length-of-stay, and 11 acute and chronic phenotypes. Measured by linear probing, the models do not provide conclusive improvement over conventional supervised baselines on low-data and low-prevalence tasks; a simple tabular baseline is often best calibrated and competitive in discrimination. The benchmark also shows that explicit temporal modeling matters more than adding data modalities, and that pre-training does not automatically transfer across institutions.

What carries the argument

The load-bearing mechanism is the FoMoH evaluation protocol itself: 14 clinically defined tasks with at-risk and case cohorts, a fixed patient split that separates pre-training and evaluation, six models standardized to 768-dimensional embeddings with roughly 120 million parameters, and linear probing — fitting a logistic regression with cross-validated ℓ2 penalty on frozen embeddings — as the common readout. The protocol measures AUROC, Brier score, and maximum subgroup gaps by sex, race, and healthcare utilization, which lets the paper separate discrimination, calibration, and fairness rather than relying on a single rank.

What would settle it

Re-run the same 14 tasks with full fine-tuning of the same six models instead of linear probing; if masked-language-model architectures such as CEHR-BERT or CORE-BEHRT then outperform the best tabular baseline on the rare phenotypes, the paper's negative conclusion would be shown to be an artifact of the probing protocol rather than a property of the representations. An audit of the patient split for pre-training/evaluation overlap would settle whether leakage is present.

Watch

Extended reading notes

Core claim

The central claim is that, at least under a frozen-embedding linear-probing evaluation, current EHR foundation models have not yet earned their promise in the settings where labels are scarcest. On common tasks such as readmission the best foundation model wins, but on rarer phenotypes the best supervised baseline frequently has the best calibration and competitive discrimination, and the gap grows in the small-data regime. The paper further claims that models with explicit temporal objectives — predicting time-to-event in MOTOR or time tokens in CEHR-GPT — do better on longer-horizon outcomes, that including laboratory values can hurt rather than help when embedding size is fixed, and that an externally pre-trained model degrades sharply on a new institution's data, partly because tokenizers fail to recognize that institution's codes.

Load-bearing premise

The conclusions depend on the assumption that reading out a foundation model's frozen embedding with a simple linear classifier is a fair test of its usefulness in the clinic, and that evaluation patients were truly excluded from pre-training; if either fails, the negative results are artifacts of the protocol.

Editorial extensions

If this is right

  • Frozen embeddings from current EHR foundation models should not be assumed to help where labeled data and outcome prevalence are low; the benchmark finds no conclusive edge over tabular baselines there.
  • Future EHR pre-training should invest in explicit temporal objectives such as time-to-event prediction or inter-event time tokens, since these correlate with the best long-horizon results in this evaluation.
  • Adding more data sources such as laboratory tests is not automatically beneficial; models trained without labs performed comparably or better under the fixed embedding budget.
  • A foundation model trained at one institution can lose much of its value at another unless tokenization and coding practices align, so transportability must be measured, not assumed.
  • Evaluations that report only AUROC can miss calibration and fairness tradeoffs; the benchmark's baselines show the largest healthcare-utilization gaps, while pre-trained models are more even across groups.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A plausible reading the authors leave implicit is that masked-language-model architectures might still shine if fine-tuned end-to-end; the paper's own limitation section concedes that fine-tuning could change the rankings, so the negative result should be read as about frozen representations, not about the models' full potential.
  • If temporal modeling is the main driver, contrastive or time-aware objectives could be combined with smaller context windows to cut inference cost, a direction the paper does not test.
  • The benchmark's cohort definitions are designed to be portable, so re-running FoMoH on other hospital systems would be a direct test of whether rare-condition gaps and transportability failures generalize; the paper only evaluates one center.
  • Since baselines show larger fairness gaps by healthcare utilization, pre-training may act as a regularizer on subgroup performance, but this would need confirmation on other datasets before it becomes a design recommendation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces FoMoH, a benchmark for structured-electronic-health-record foundation models using 14 clinically motivated tasks (11 phenotypes and 3 patient outcomes) defined on roughly 5.3 million patients from Columbia University Irving Medical Center, standardized to MEDS and OMOP-compatible formats. It evaluates six foundation models (CEHR-BERT, CEHR-GPT, CORE-BEHRT, LLAMA, MAMBA, MOTOR), a transported Stanford-trained MAMBA, and tabular baselines (FEMR and MEDS-TAB with logistic regression, LightGBM, and XGBoost), all compared through local pre-training followed by linear probing. Metrics include AUROC, Brier score, subgroup fairness gaps, and inference FLOPs. The central findings are that foundation model embeddings do not provide conclusive improvement over supervised baselines for low-data and low-prevalence tasks, that explicit temporal modeling appears more valuable than adding laboratory modalities, and that direct transport of a pre-trained model across institutions degrades performance.

Significance. If the findings hold, FoMoH is a valuable resource: it contributes a standardized, ACES/MEDS-compatible task suite with a fixed patient split aimed at preventing leakage, and it broadens EHR foundation-model evaluation beyond AUROC to calibration, fairness, and computational cost. The negative result on low-data and low-prevalence tasks is an important corrective to the prevailing narrative of EHR foundation model advantage, and the observation that temporal encoding matters more than additional modalities is a concrete, testable design hypothesis. The paper also ships a public benchmark and task definitions, which strengthens reproducibility. The main limitation is that the conclusions are tied to a linear-probing protocol and to several design choices that are not fully isolated, so the strength of the claims needs to be tempered accordingly.

major comments (3)
  1. [Section 6 / 6.1 / 4.2] The central claim that "foundation models do not provide conclusive improvement over supervised baselines for low-data and low-prevalence tasks" is stated in the abstract and in Section 6 without the qualifier "under linear probing." All foundation model results are obtained by fitting a logistic regression on frozen embeddings (Section 4.2), and Section 6.1 explicitly concedes that "fine-tuning strategies may result in conclusions different from those presented in this work." Because the claimed failure regime is precisely the setting where a frozen linear probe is least likely to exploit task-adapted representations, the headline should either be qualified throughout, or the authors should add a fine-tuning comparison on a subset of low-prevalence tasks (e.g., CLL and celiac disease) to test whether the negative result is an artifact of the probing protocol.
  2. [Appendix D.1 / Tables 1 and 14] The input code vocabulary is not held constant across models. CEHR-BERT and CEHR-GPT are trained and evaluated on OMOP standard concept codes, whereas the other foundation models and the FEMR/MEDS-TAB baselines operate on MEDS source concept codes such as ICD-9/10. This choice is intended to match the original codebases, but it means that cross-model comparisons, for example the CEHR-GPT versus MAMBA differences reported in Section 5, conflate model design with code representation. The manuscript should state this confound wherever cross-model conclusions are drawn and, ideally, include an ablation with a common code vocabulary for at least one model pair to assess its impact.
  3. [Section 4 / Appendix F.4 / Table 2] The construction of "Best Baseline" is not defined. Appendix F.4 states that "all previous tables and figures reflect the best performance across these different models," but it does not specify whether the best baseline is selected on the 10% tuning split or on the test set. If selection is based on test-set performance, the baseline is an oracle and the central "no conclusive improvement" claim is biased in favor of the baselines. Please specify the selection procedure, and if test-based selection is used, report each baseline separately and identify the baseline selected on the tuning split.
minor comments (6)
  1. [Abstract] There is a typo in "across14 clinically relevant tasks"; a space is missing between "across" and "14."
  2. [Figure 2 caption / Section 4] The Figure 2 caption says the top row reflects the smallest amount of data used for training the linear probing, "i.e., 100 points," while Section 4 states that the training set is capped at 100,000 records. Please clarify whether "100 points" refers to a specific task, to positive examples, or to the smallest task size, and reconcile the numbers.
  3. [Table 6 / Appendix D.2] The last row of Table 6 is labeled "FEMR" but the surrounding text describes a decoder that uses RoPE and sinusoidal age embeddings; this appears to be a typo for MOTOR. Please correct the label or the accompanying text.
  4. [Section 3.3] In the CORE-BEHRT paragraph, "Contrarily to CEHR-BEHRT" should read "CEHR-BERT."
  5. [Section 5] The sentence "In our analysis of available data sources (Appendix C), we find that lab tests are, on average, more prevalent" is confusing; the surrounding argument is that lab tests are not informative. Consider rewording to "more numerous but not more informative" or similar.
  6. [Section 4 / Appendix B] Because the low-prevalence argument is central to the paper, consider reporting the prevalence and cohort size for every task in the main text rather than only in the appendix, so that readers can gauge the reliability of the AUROC and Brier estimates for rare phenotypes.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the benchmark evaluates held-out downstream tasks with linear probing against independent baselines, and the acknowledged scope limitation is not a circular reduction.

full rationale

This paper is an empirical benchmark rather than a derivation, so there is no chain in which a fitted parameter is renamed as a prediction or in which a result reduces by construction to an input. Downstream evaluation uses a fixed patient split (60% train, 10% tuning, 30% evaluation), linear probing with cross-validated lambda selection, and bootstrapped confidence intervals. All foundation models, including the authors' own CEHR-BERT and CEHR-GPT, are retrained locally on CUMC data and compared against independent tabular baselines (FEMR, MEDS-TAB) and external architectures (LLAMA, MAMBA, MOTOR), so the central comparison is externally grounded rather than forced by any fitted value. The paper's statement that 'fine-tuning strategies may result in conclusions different from those presented in this work' (Section 6.1) is an explicit scope restriction on the linear-probing protocol, not a hidden circular step; it qualifies the claim but does not make the claim equivalent to its inputs. The use of same-institution models is a standard benchmarking choice and does not constitute self-citation load-bearing evidence, because the models' performance is measured on held-out outcomes rather than inferred from the cited papers. No specific equation or definition is found that reduces to itself, so the appropriate finding is no significant circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The benchmark's conclusions rest on clinical phenotype definitions, data standardization choices, and a probing protocol rather than on fitted physical parameters. The thresholds and cohort rules are transparently documented but are hand-chosen design decisions that shape all reported numbers.

free parameters (5)
  • Context length coverage threshold = 99%
    Context length per model was chosen as the smallest power of two covering at least 99% of visits; this hand-chosen threshold affects how much history each model sees.
  • Linear probing training cap = 100,000 records
    Training set for each task is capped at 100,000 records to equalize data across tasks; this choice affects low-data regime results.
  • Evaluation set cap = 50,000 records
    Evaluation set is capped at 50,000 records; this affects confidence interval width and which patients are evaluated.
  • Phenotype prediction window = 1 year
    All phenotypes use a 1-year prediction window; this is a clinical design choice, not fitted to data.
  • Healthcare utilization tertiles = Empirical tertiles of encounters per year
    Fairness strata are defined as empirical tertiles of healthcare utilization; this grouping choice affects fairness gap estimates.
assumptions (5)
  • domain assumption EHR data in OMOP CDM and MEDS faithfully represent patient health trajectories.
    The entire benchmark assumes the standardized EHR data capture clinically relevant history; errors in coding or missingness are not modeled.
  • domain assumption Phenotype algorithms from OHDSI, PheKB, and clinical risk factors are valid proxies for ground-truth disease status.
    Case and at-risk cohorts are defined by these algorithms in Appendix B; misclassification would bias all performance estimates.
  • domain assumption ICD-9 to ICD-10 mapping via GEM is adequate for harmonizing diagnoses.
    Section 3.1 maps all ICD-9 codes to ICD-10 using GEM, dropping unmapped codes; mapping errors could distort tokens and outcomes.
  • ad hoc to paper Linear probing of frozen embeddings reflects downstream clinical utility.
    Section 4.2 evaluates all models with linear probing only; the paper itself notes fine-tuning could change conclusions in Section 6.1.
  • ad hoc to paper Two years of observation before prediction time is sufficient baseline history.
    All outcome tasks require at least two years of observation (Section 3.2); this threshold is chosen by hand and affects which patients are included.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FoMoH: A clinically meaningful foundation model evaluation for structured electronic health records." pith.science (2026). https://pith.science/paper/4BOOTXZC

@misc{pith2026250516941,
  author       = {Pith},
  title        = {Pith review of: FoMoH: A clinically meaningful foundation model evaluation for structured electronic health records},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4BOOTXZC}},
  note         = {Machine review of arXiv:2505.16941}
}
read the original abstract

Foundation models hold significant promise in healthcare, given their capacity to extract meaningful representations independent of downstream tasks. This property has enabled state-of-the-art performance across several clinical applications trained on structured electronic health record (EHR) data, even in settings with limited labeled data, a prevalent challenge in healthcare. However, there is little consensus on these models' potential for clinical utility due to the lack of desiderata of comprehensive and meaningful tasks and sufficiently diverse evaluations to characterize the benefit over conventional supervised learning. To address this gap, we propose a suite of clinically meaningful tasks spanning patient outcomes, early prediction of acute and chronic conditions, including desiderata for robust evaluations. We evaluate state-of-the-art foundation models on EHR data consisting of 5 million patients from Columbia University Irving Medical Center (CUMC), a large urban academic medical center in New York City, across 14 clinically relevant tasks. We measure overall accuracy, calibration, and subpopulation performance to surface tradeoffs based on the choice of pre-training, tokenization, and data representation strategies. Our study aims to advance the empirical evaluation of structured EHR foundation models and guide the development of future healthcare foundation models.

Figures

Figures reproduced from arXiv: 2505.16941 by the authors.

Figure 1
Figure 1. FoMoH is an evaluation of state-of-the-art structured EHR foundation models using (i) clinically meaningful and diverse downstream tasks, (ii) patient cohorts derived to prevent data leakage, (iii) augmented evaluation beyond overall discrimination, such as calibration, and fairness along clinically relevant factors such as healthcare utilization. unique challenges associated with EHR, such as temporality [56, 67] a… view at source ↗
Figure 2
Figure 2. Average linear probing results as a function of the prevalence of positive labels for each task. The leftmost column shows discriminative performance, the middle column shows model calibration, and the right column shows fairness by healthcare utilization. The shaded area represents the bootstrapped standard deviation. The top row reflects the smallest amount of data used for training the linear probing, i.e., 100 p… view at source ↗
Figure 3
Figure 3. The proposed phenotypes are structured using consistent temporal components: (observation entry, index, and cohort exit) and clear inclusion criteria for the at-risk and case cohorts. (a) Example of an individual included in the case cohort, as their case-cohort inclusion event occurs during the prediction window. (b-c) Examples of individuals included in the at-risk, but not the case cohort, since they have an at-r… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p024_4.png]
Figure 5
Figure 5. Figure 5: In order to compare the density of data at each institution, we measure the number of occurrences of each data type (a. conditions, b. medications, c. procedures, d. lab tests) per year of observation for each patient. Unlike with the healthcare utilization metric, we …
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p026_6.png]
Figure 7
Figure 7. Figure 7: Percentage of the population covered given different context windows. D.4 Other hyperparameters All other pre-training hyperparameters are described in [PITH_FULL_IMAGE:figures/full_fig_p026_7.png]
Figure 8
Figure 8. Figure 8: Discriminative performances and calibration results for all outcomes for increasing number of training points (Shaded area represents bootstrapped standard deviation). 30 [PITH_FULL_IMAGE:figures/full_fig_p030_8.png]
Figure 9
Figure 9. Figure 9: Discriminative performances and calibration results for all outcomes for increasing number of training points (Shaded area represents bootstrapped standard deviation). 31 [PITH_FULL_IMAGE:figures/full_fig_p031_9.png]
Figure 10
Figure 10. Figure 10: Discriminative performances and calibration results for all phenotypes for increasing number of training points (Shaded area represents bootstrapped standard deviation). 32 [PITH_FULL_IMAGE:figures/full_fig_p032_10.png]
Figure 11
Figure 11. Figure 11: Discriminative performances and calibration results for all phenotypes for increasing number of training points (Shaded area represents bootstrapped standard deviation) - Continued. 33 [PITH_FULL_IMAGE:figures/full_fig_p033_11.png]
Figure 12
Figure 12. Figure 12: Discriminative performances and calibration results for all phenotypes for increasing number of training points (Shaded area represents bootstrapped standard deviation). 34 [PITH_FULL_IMAGE:figures/full_fig_p034_12.png]
Figure 13
Figure 13. Figure 13: Discriminative performances and calibration results for all phenotypes for increasing number of training points (Shaded area represents bootstrapped standard deviation) - Continued. 35 [PITH_FULL_IMAGE:figures/full_fig_p035_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

89 extracted references · 53 canonical work pages

  1. [1]

    Biases in electronic health record data due to processes within the healthcare system: retrospective observational study

    Denis Agniel, Isaac S Kohane, and Griffin M Weber. Biases in electronic health record data due to processes within the healthcare system: retrospective observational study. Bmj, 361, 2018

    2018

  2. [2]

    Risk factors for chronic lymphocytic leukemia

    American Cancer Society. Risk factors for chronic lymphocytic leukemia. https://www.cancer.org/cancer/types/chronic-lymphocytic-leukemia/ causes-risks-prevention/risk-factors.html , 2024. Accessed: 2025-05-05

    2024

  3. [3]

    McDermott, Jungwoo Oh, Tom Pollard, Nigam Shah, Ethan Steinberg, Michael Wornow, and Robin van de Water

    Bert Arnrich, Edward Choi, Jason Alan Fries, Matthew B.A. McDermott, Jungwoo Oh, Tom Pollard, Nigam Shah, Ethan Steinberg, Michael Wornow, and Robin van de Water. Medical event data standard (MEDS): Facilitating machine learning for health. In ICLR 2024 Workshop on Learning from Time Series For Health, 2024. URL https://openreview.net/forum? id=IsHy2ebjIG

    2024

  4. [4]

    Fairness and machine learning: Limita- tions and opportunities

    Solon Barocas, Moritz Hardt, and Arvind Narayanan. Fairness and machine learning: Limita- tions and opportunities. MIT press, 2023

    2023

  5. [5]

    Characterizing and managing missing structured data in electronic health records: data analysis

    Brett K Beaulieu-Jones, Daniel R Lavage, John W Snyder, Jason H Moore, Sarah A Pendergrass, and Christopher R Bauer. Characterizing and managing missing structured data in electronic health records: data analysis. JMIR medical informatics, 6(1):e8960, 2018

    2018

  6. [6]

    Machine learning for patient risk stratification: standing on, or looking over, the shoulders of clinicians? NPJ digital medicine, 4(1):62, 2021

    Brett K Beaulieu-Jones, William Yuan, Gabriel A Brat, Andrew L Beam, Griffin Weber, Marshall Ruffin, and Isaac S Kohane. Machine learning for patient risk stratification: standing on, or looking over, the shoulders of clinicians? NPJ digital medicine, 4(1):62, 2021

    2021

  7. [7]

    Hospitalizations for chronic disease and acute conditions in the time of covid-19

    Saul Blecker, Simon A Jones, Christopher M Petrilli, Andrew J Admon, Himali Weerahandi, Fritz Francois, and Leora I Horwitz. Hospitalizations for chronic disease and acute conditions in the time of covid-19. JAMA internal medicine, 181(2):269–271, 2021

    2021

  8. [8]

    On the opportunities and risks of foundation models

    Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021

    arXiv 2021

Show all 89 references
  1. [9]

    Risk factors for stroke

    Centers for Disease Control and Prevention. Risk factors for stroke. https://www.cdc.gov/ stroke/risk-factors/index.html, 2024. Accessed: 2025-05-05

    2024

  2. [10]

    Multimodal clinical benchmark for emergency care (mc-bec): A comprehensive benchmark for evaluating foundation models in emergency medicine

    Emma Chen, Aman Kansal, Julie Chen, Boyang Tom Jin, Julia Reisler, David E Kim, and Pranav Rajpurkar. Multimodal clinical benchmark for emergency care (mc-bec): A comprehensive benchmark for evaluating foundation models in emergency medicine. Advances in Neural Information Pro...

    2023

  3. [11]

    Probabilistic machine learning for healthcare

    Irene Y* Chen, Shalmali Joshi*, Marzyeh Ghassemi, and Rajesh Ranganath. Probabilistic machine learning for healthcare. Annual Review of Biomedical Data Science, 4, 2020

    2020

  4. [12]

    Chen, Emma Pierson, Sherri Rose, Shalmali Joshi, Kadija Ferryman, and Marzyeh Ghassemi

    Irene Y . Chen, Emma Pierson, Sherri Rose, Shalmali Joshi, Kadija Ferryman, and Marzyeh Ghassemi. Ethical machine learning in healthcare. Annual Review of Biomedical Data Science ,

  5. [13]

    Xgboost: A scalable tree boosting system

    Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, page 785–794. ACM, August 2016. doi: 10.1145/2939672.2939785. URL http://dx.doi.org/10.114...

    2016

  6. [14]

    Metabolic dysfunction-associated steatotic liver disease increases cardiovascular disease risk in young adults

    Goh Eun Chung, Su Jong Yu, Jeong-Ju Yoo, Yuri Cho, Kyu na Lee, Dong Wook Shin, Yoon Jun Kim, Jung-Hwan Yoon, Kyungdo Han, and Eun Ju Cho. Metabolic dysfunction-associated steatotic liver disease increases cardiovascular disease risk in young adults. Scientific Re- ports, 15:57...

    2025 doi

  7. [15]

    InstructBLIP: Towards general-purpose vision-language models with instruction tuning

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. InstructBLIP: Towards general-purpose vision-language models with instruction tuning. In Thirty-seventh Conference on Neural Information Processing Systems, 2...

    2023

  8. [16]

    A decoder-only foundation model for time-series forecasting

    Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. A decoder-only foundation model for time-series forecasting. In Forty-first International Conference on Machine Learning, 2024

    2024

  9. [17]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human langu...

    2019

  10. [18]

    Ehrmamba: Towards generalizable and scalable foundation models for electronic health records

    Adibvafa Fallahpour, Mahshid Alinoori, Wenqian Ye, Xu Cao, Arash Afkanpour, and Amrit Krishnan. Ehrmamba: Towards generalizable and scalable foundation models for electronic health records. arXiv preprint arXiv:2405.14567, 2024

    2024 arXiv

  11. [19]

    Prevalence and inci- dence measures for schizophrenia among commercial health insurance and medicaid enrollees

    Molly T Finnerty, Atif Khan, Kai You, Rui Wang, Gyojeong Gu, Deborah Layman, Qingx- ian Chen, Noémie Elhadad, Shalmali Joshi, Paul S Appelbaum, et al. Prevalence and inci- dence measures for schizophrenia among commercial health insurance and medicaid enrollees. Schizophrenia,...

    2024

  12. [20]

    A foundation model of transcription across human cell types

    Xi Fu, Shentong Mo, Alejandro Buendia, Anouchka P Laurent, Anqi Shao, Maria del Mar Alvarez-Torres, Tianji Yu, Jimin Tan, Jiayu Su, Romella Sagatelian, et al. A foundation model of transcription across human cell types. Nature, pages 1–9, 2025

    2025

  13. [21]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  14. [22]

    Mamba: Linear-time sequence modeling with selective state spaces

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2023

    2023 arXiv

  15. [23]

    Foundation models in bioinformatics

    Fei Guo, Renchu Guan, Yaohang Li, Qi Liu, Xiaowo Wang, Can Yang, and Jianxin Wang. Foundation models in bioinformatics. National Science Review, page nwaf028, 2025

    2025

  16. [24]

    Foundation model for advancing healthcare: challenges, opportunities and future directions

    Yuting He, Fuxiang Huang, Xinrui Jiang, Yuxiang Nie, Minghao Wang, Jiguang Wang, and Hao Chen. Foundation model for advancing healthcare: challenges, opportunities and future directions. IEEE Reviews in Biomedical Engineering, 2024

    2024

  17. [25]

    Large language models are powerful ehr encoders

    Stefan Hegselmann, Georg von Arnim, Tillmann Rheude, Noel Kronenberg, David Sontag, Gerhard Hindricks, Roland Eils, and Benjamin Wild. Large language models are powerful ehr encoders. arXiv preprint arXiv:2502.17403, 2025

    2025 arXiv

  18. [26]

    Foundation models and fair use

    Peter Henderson, Xuechen Li, Dan Jurafsky, Tatsunori Hashimoto, Mark A Lemley, and Percy Liang. Foundation models and fair use. Journal of Machine Learning Research, 24(400):1–79, 2023

    2023

  19. [27]

    Feasibility of large-scale observa- tional cancer research using the ohdsi network—aim 2 findings

    George Hripcsak, RuiJun Chen, and Thomas Falconer. Feasibility of large-scale observa- tional cancer research using the ohdsi network—aim 2 findings. https://www.ohdsi.org/ wp-content/uploads/2015/04/NCI-FinalPresentation-OHDSI.pdf . Accessed: 2025- 05-06

    2015

  20. [28]

    Observational health data sciences and informatics (ohdsi): opportunities for observational researchers

    George Hripcsak, Jon D Duke, Nigam H Shah, Christian G Reich, V ojtech Huser, Martijn J Schuemie, Marc A Suchard, Rae Woong Park, Ian Chi Kei Wong, Peter R Rijnbeek, et al. Observational health data sciences and informatics (ohdsi): opportunities for observational researchers....

    2015

  21. [29]

    A foundation model for clinician-centered drug repurposing

    Kexin Huang, Payal Chandak, Qianwen Wang, Shreyas Havaldar, Akhil Vaid, Jure Leskovec, Girish N Nadkarni, Benjamin S Glicksberg, Nils Gehlenborg, and Marinka Zitnik. A foundation model for clinician-centered drug repurposing. Nature Medicine, 30(12):3601–3613, 2024

    2024

  22. [30]

    Recent epidemiology and risk factors of nonalcoholic fatty liver disease

    Youn Huh, Yoon Jeong Cho, and Ga Eun Nam. Recent epidemiology and risk factors of nonalcoholic fatty liver disease. Journal of Obesity & Metabolic Syndrome, 31(1):17–27, 2022. doi: 10.7570/jomes22021. URL https://www.jomes.org/journal/view.html?doi=10. 7570/jomes22021. 12

    2022 doi

  23. [31]

    Genhpf: General healthcare predictive framework for multi-task multi-source learning

    Kyunghoon Hur, Jungwoo Oh, Junu Kim, Jiyoun Kim, Min Jae Lee, Eunbyeol Cho, Seong- Eun Moon, Young-Hak Kim, Louis Atallah, and Edward Choi. Genhpf: General healthcare predictive framework for multi-task multi-source learning. IEEE Journal of Biomedical and Health Informatics, ...

    2023

  24. [32]

    Clinical Presence: Impact on Predictive Modelling and Algorithmic Fairness

    Vincent Jeanselme. Clinical Presence: Impact on Predictive Modelling and Algorithmic Fairness. PhD thesis, University of Cambridge, 2024

    2024

  25. [33]

    Review of language models for survival analysis

    Vincent Jeanselme, Nikita Agarwal, and Chen Wang. Review of language models for survival analysis. In AAAI 2024 Spring Symposium on Clinical Foundation Models, 2024

    2024

  26. [34]

    Event-based contrastive learning for medical time series

    Hyewon Jeong, Nassim Oufattole, Matthew Mcdermott, Aparna Balagopalan, Bryan Jangeesingh, Marzyeh Ghassemi, and Collin Stultz. Event-based contrastive learning for medical time series. arXiv preprint arXiv:2312.10308, 2023

    2023 arXiv

  27. [35]

    Ai as an intervention: improving clinical outcomes relies on a causal approach to ai development and validation

    Shalmali Joshi, Iñigo Urteaga, Wouter AC van Amsterdam, George Hripcsak, Pierre Elias, Benjamin Recht, Noémie Elhadad, James Fackler, Mark P Sendak, Jenna Wiens, et al. Ai as an intervention: improving clinical outcomes relies on a causal approach to ai development and validat...

    2025

  28. [36]

    Scaling laws for neural language models

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020

    2001 arXiv

  29. [37]

    Time2Vec: Learning a Vector Representation of Time, July 2019

    Seyed Mehran Kazemi, Rishab Goel, Sepehr Eghbali, Janahan Ramanan, Jaspreet Sahota, Sanjay Thakur, Stella Wu, Cathal Smyth, Pascal Poupart, and Marcus Brubaker. Time2Vec: Learning a Vector Representation of Time, July 2019. URL http://arxiv.org/abs/1907.05321. arXiv:1907.05321 [cs]

    2019 arXiv

  30. [38]

    Lightgbm: a highly efficient gradient boosting decision tree

    Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. Lightgbm: a highly efficient gradient boosting decision tree. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, page 3149–3157...

    2017

  31. [39]

    Meds decentralized, extensible validation (meds-dev) benchmark: Establishing reproducibility and comparability in ml for health

    Aleksia Kolo, Chao Pang, Edward Choi, Ethan Steinberg, Hyewon Jeong, Jack Gallifant, Jason A Fries, Jeffrey N Chiang, Jungwoo Oh, Justin Xu, et al. Meds decentralized, extensible validation (meds-dev) benchmark: Establishing reproducibility and comparability in ml for health. 2024

    2024

  32. [40]

    Medgpt: Medical concept prediction from clinical narratives

    Zeljko Kraljevic, Anthony Shek, Daniel Bean, Rebecca Bendayan, James Teo, and Richard Dobson. Medgpt: Medical concept prediction from clinical narratives. arXiv preprint arXiv:2107.03134, 2021

    2021 arXiv

  33. [41]

    Foresight—a generative pretrained transformer for modelling of patient timelines using electronic health records: a retrospective modelling study

    Zeljko Kraljevic, Dan Bean, Anthony Shek, Rebecca Bendayan, Harry Hemingway, Joshua Au Yeung, Alexander Deng, Alfred Baston, Jack Ross, Esther Idowu, et al. Foresight—a generative pretrained transformer for modelling of patient timelines using electronic health records: a retr...

    2024

  34. [42]

    Comparative effectiveness of medical concept embedding for feature engineering in phenotyping

    Junghwan Lee, Cong Liu, Jae Hyun Kim, Alex Butler, Ning Shang, Chao Pang, Karthik Natarajan, Patrick B Ryan, Casey Ta, and Chunhua Weng. Comparative effectiveness of medical concept embedding for feature engineering in phenotyping. JAMIA Open, 4:2, 2021

    2021

  35. [43]

    Emergency department decision support using clinical pseudo-notes

    Simon A Lee, Sujay Jain, Alex Chen, Kyoka Ono, Jennifer Fang, Akos Rudas, and Jeffrey N Chiang. Emergency department decision support using clinical pseudo-notes. arXiv preprint arXiv:2402.00160, 2024

    2024 arXiv

  36. [44]

    Behrt: transformer for electronic health records

    Yikuan Li, Shishir Rao, José Roberto Ayala Solares, Abdelaali Hassaine, Rema Ramakrishnan, Dexter Canoy, Yajie Zhu, Kazem Rahimi, and Gholamreza Salimi-Khorshidi. Behrt: transformer for electronic health records. Scientific reports, 10(1):7155, 2020

    2020

  37. [45]

    Prevalence and pattern of acute and chronic multimorbidity across all body systems and age groups in primary health care

    Michael Linden, Ulrike Linden, David Goretzko, and Jochen Gensichen. Prevalence and pattern of acute and chronic multimorbidity across all body systems and age groups in primary health care. Scientific Reports, 12(1):272, 2022. 13

    2022

  38. [46]

    A visual-language foundation model for computational pathology

    Ming Y Lu, Bowen Chen, Drew FK Williamson, Richard J Chen, Ivy Liang, Tong Ding, Guil- laume Jaume, Igor Odintsov, Long Phi Le, Georg Gerber, et al. A visual-language foundation model for computational pathology. Nature Medicine, 30(3):863–874, 2024

    2024

  39. [47]

    A benchmarking crisis in biomedical machine learning

    Faisal Mahmood. A benchmarking crisis in biomedical machine learning. Nature Medicine, pages 1–1, 2025

    2025

  40. [48]

    A closer look at benchmarking self-supervised pre-training with image classification

    Markus Marks, Manuel Knott, Neehar Kondapaneni, Elijah Cole, Thijs Defraeye, Fernando Perez-Cruz, and Pietro Perona. A closer look at benchmarking self-supervised pre-training with image classification. International Journal of Computer Vision, pages 1–13, 2025

    2025

  41. [49]

    Event stream gpt: a data pre-processing and modeling library for generative, pre-trained transformers over continuous- time sequences of complex events

    Matthew McDermott, Bret Nestor, Peniel Argaw, and Isaac S Kohane. Event stream gpt: a data pre-processing and modeling library for generative, pre-trained transformers over continuous- time sequences of complex events. Advances in Neural Information Processing Systems, 36: 243...

    2023

  42. [50]

    Event stream gpt: a data pre-processing and modeling library for generative, pre-trained transformers over continuous- time sequences of complex events

    Matthew McDermott, Bret Nestor, Peniel Argaw, and Isaac S Kohane. Event stream gpt: a data pre-processing and modeling library for generative, pre-trained transformers over continuous- time sequences of complex events. Advances in Neural Information Processing Systems, 36, 2024

    2024

  43. [51]

    Reproducibility in machine learning for health research: Still a ways to go

    Matthew BA McDermott, Shirly Wang, Nikki Marinsek, Rajesh Ranganath, Luca Foschini, and Marzyeh Ghassemi. Reproducibility in machine learning for health research: Still a ways to go. Science Translational Medicine, 13(586):eabb1655, 2021

    2021

  44. [52]

    Foundation models for generalist medical artificial intelligence

    Michael Moor, Oishi Banerjee, Zahra Shakeri Hossein Abad, Harlan M Krumholz, Jure Leskovec, Eric J Topol, and Pranav Rajpurkar. Foundation models for generalist medical artificial intelligence. Nature, 616(7956):259–265, 2023

    2023

  45. [53]

    Risk factors for type 2 diabetes

    National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK). Risk factors for type 2 diabetes. https://www.niddk.nih.gov/health-information/diabetes/overview/ risk-factors-type-2-diabetes , 2025. Accessed: 2025-05-05

    2025

  46. [54]

    Core-behrt: A carefully optimized and rigorously evaluated behrt

    Mikkel Odgaard, Kiril Vadimovic Klein, Sanne Møller Thysen, Espen Jimenez-Solem, Martin Sillesen, and Mads Nielsen. Core-behrt: A carefully optimized and rigorously evaluated behrt. arXiv preprint arXiv:2404.15201, 2024

    2024 arXiv

  47. [55]

    Stultz, and Matthew B

    Nassim Oufattole, Teya Bergamaschi, Aleksia Kolo, Hyewon Jeong, Hanna Gaggin, Collin M. Stultz, and Matthew B. A. McDermott. Meds-tab: Automated tabularization and baseline methods for meds datasets, 2024. URL https://arxiv.org/abs/2411.00200

    2024 arXiv

  48. [56]

    Cehr-bert: Incorporating temporal information from structured ehr data to improve prediction tasks

    Chao Pang, Xinzhuo Jiang, Krishna S Kalluri, Matthew Spotnitz, RuiJun Chen, Adler Perotte, and Karthik Natarajan. Cehr-bert: Incorporating temporal information from structured ehr data to improve prediction tasks. In Machine Learning for Health, pages 239–260. PMLR, 2021

    2021

  49. [57]

    Cehr-gpt: Generating electronic health records with chronological patient timelines

    Chao Pang, Xinzhuo Jiang, Nishanth Parameshwar Pavinkurve, Krishna S Kalluri, Elise L Minto, Jason Patterson, Linying Zhang, George Hripcsak, Gamze Gürsoy, Noémie Elhadad, et al. Cehr-gpt: Generating electronic health records with chronological patient timelines. arXiv preprin...

    2024 arXiv

  50. [58]

    GloVe: Global vectors for word representation

    Jeffrey Pennington, Richard Socher, and Christopher Manning. GloVe: Global vectors for word representation. In Alessandro Moschitti, Bo Pang, and Walter Daelemans, editors, Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages 1...

    2014 doi

  51. [59]

    Improving language understanding by generative pre-training

    Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understanding by generative pre-training. 2018

    2018

  52. [60]

    Med-bert: pretrained contex- tualized embeddings on large-scale structured electronic health records for disease prediction

    Laila Rasmy, Yang Xiang, Ziqian Xie, Cui Tao, and Degui Zhi. Med-bert: pretrained contex- tualized embeddings on large-scale structured electronic health records for disease prediction. NPJ digital medicine, 4(1):86, 2021. 14

    2021

  53. [61]

    Zero shot health trajectory prediction using transformer

    Pawel Renc, Yugang Jia, Anthony E Samir, Jaroslaw Was, Quanzheng Li, David W Bates, and Arkadiusz Sitek. Zero shot health trajectory prediction using transformer. npj Digital Medicine, 7(1):256, 2024

    2024

  54. [62]

    Creation and adoption of large language models in medicine

    Nigam H Shah, David Entwistle, and Michael A Pfeffer. Creation and adoption of large language models in medicine. Jama, 330(9):866–869, 2023

    2023

  55. [63]

    Advancing the science for active surveillance: rationale and design for the observational medical outcomes partnership

    Paul E Stang, Patrick B Ryan, Judith A Racoosin, J Marc Overhage, Abraham G Hartzema, Christian Reich, Emily Welebob, Thomas Scarnecchia, and Janet Woodcock. Advancing the science for active surveillance: rationale and design for the observational medical outcomes partnership....

    2010

  56. [64]

    Fries, Conor K

    Ethan Steinberg, Ken Jung, Jason A. Fries, Conor K. Corbin, Stephen R. Pfohl, and Nigam H. Shah. Language models are an effective patient representation learning technique for electronic health record data, 2020. URL https://arxiv.org/abs/2001.05295

    2020 arXiv

  57. [65]

    Language models are an effective representation learning technique for electronic health record data

    Ethan Steinberg, Ken Jung, Jason A Fries, Conor K Corbin, Stephen R Pfohl, and Nigam H Shah. Language models are an effective representation learning technique for electronic health record data. Journal of biomedical informatics, 113:103637, 2021

    2021

  58. [66]

    meds_reader: A fast and efficient ehr processing library

    Ethan Steinberg, Michael Wornow, Suhana Bedi, Jason Alan Fries, Matthew McDermott, and Nigam H Shah. meds_reader: A fast and efficient ehr processing library. arXiv preprint arXiv:2409.09095, 2024

    2024 arXiv

  59. [67]

    MOTOR: A time-to-event foundation model for structured medical records

    Ethan Steinberg, Yizhe Xu, Jason Alan Fries, and Nigam Shah. MOTOR: A time-to-event foundation model for structured medical records. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=NialiwI2V6

    2024

  60. [68]

    Multimodal medical code tokenizer

    Xiaorui Su, Shvat Messica, Yepeng Huang, Ruth Johnson, Lukas Fesser, Shanghua Gao, Faryad Sahneh, and Marinka Zitnik. Multimodal medical code tokenizer. arXiv preprint arXiv:2502.04397, 2025

    2025 arXiv

  61. [70]

    Suchard, Martijn J

    Marc A. Suchard, Martijn J. Schuemie, Harlan M. Krumholz, Seng Chan You, RuiJun Chen, Nicole Pratt, Christian G. Reich, Jon Duke, David Madigan, George Hripcsak, and Patrick B. Ryan. Comprehensive comparative effectiveness and safety of first-line antihypertensive drug classes...

    2019 doi

  62. [71]

    Marc A Suchard, Martijn J Schuemie, Harlan M Krumholz, Seoyoung C You, Ren Chen, Nicole Pratt, Christian Reich, Patrick Ryan, and George Hripcsak. Large-scale evidence generation and evaluation across a network of databases for type 2 diabetes mellitus (legend-t2dm): a protoco...

    2021 doi

  63. [72]

    Phevaluator: Development and evaluation of a phenotype algorithm evaluator

    Joel N Swerdel, George Hripcsak, and Patrick B Ryan. Phevaluator: Development and evaluation of a phenotype algorithm evaluator. Journal of Biomedical Informatics, 97:103258, 2019

    2019

  64. [73]

    Swerdel, Darmendra Ramcharran, and Jill Hardin

    Joel N. Swerdel, Darmendra Ramcharran, and Jill Hardin. Using a data-driven approach for the development and evaluation of phenotype algorithms for systemic lupus erythematosus. PLOS ONE, 18(2):e0281929, 2023. ISSN 1932-6203. doi: 10.1371/journal.pone.0281929

    2023 doi

  65. [74]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023. 15

    2023 arXiv

  66. [75]

    Harmonizing foundation models in healthcare: A comprehensive survey of their roles, relationships, and impact in artificial intelligence’s advancing terrain

    Mohan Timilsina, Samuele Buosi, Muhammad Asif Razzaq, Rafiqul Haque, Conor Judge, and Edward Curry. Harmonizing foundation models in healthcare: A comprehensive survey of their roles, relationships, and impact in artificial intelligence’s advancing terrain. Computers in Biolog...

    2025

  67. [76]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  68. [77]

    Osteoporosis risk factors

    UC San Diego Health. Osteoporosis risk factors. https://health.ucsd.edu/care/ endocrinology-diabetes/osteoporosis/risk-factors/ , 2024. Accessed: 2025-05- 05

    2024

  69. [78]

    Diagnosis of acute myocardial infarction

    UpToDate. Diagnosis of acute myocardial infarction. https://www.uptodate.com/ contents/diagnosis-of-acute-myocardial-infarction?search=AMI&source= search_result&selectedTitle=2~150&usage_type=default&display_rank=2, . Accessed: 2025-05-06

    2025

  70. [79]

    Diagnosis of celiac disease in adults

    UpToDate. Diagnosis of celiac disease in adults. https://www.uptodate.com/contents/ diagnosis-of-celiac-disease-in-adults , . Accessed: 2025-05-06

    2025

  71. [80]

    Overview of hypertension in adults

    UpToDate. Overview of hypertension in adults. https://www.uptodate.com/ contents/overview-of-hypertension-in-adults?search=prevalence%20of% 20hypertension&source=search_result&selectedTitle=2%7E150&usage_type= default&display_rank=2#H8, . Accessed: 2025-05-06

    2025

  72. [81]

    Hypertension in adults: Epidemiology

    UpToDate. Hypertension in adults: Epidemiology. https://www.uptodate.com/ contents/the-prevalence-and-control-of-hypertension-in-adults?search= hypertension&topicRef=3852&source=see_link#H1773280799, . Accessed: 2025-05- 06

    2025

  73. [82]

    Clinical manifestations, diagnosis, and staging of ex- ocrine pancreatic cancer

    UpToDate. Clinical manifestations, diagnosis, and staging of ex- ocrine pancreatic cancer. https://www.uptodate.com/contents/ clinical-manifestations-diagnosis-and-staging-of-exocrine-pancreatic-cancer? search=pancreatic%20cancer&source=search_result&selectedTitle=1~150& usage...

    2025

  74. [83]

    Systemic lupus erythematosus in adults: Clinical man- ifestations and diagnosis

    UpToDate. Systemic lupus erythematosus in adults: Clinical man- ifestations and diagnosis. https://www.uptodate.com/contents/ systemic-lupus-erythematosus-in-adults-clinical-manifestations-and-diagnosis? search=lupus&source=search_result&selectedTitle=1~150&usage_type= default...

    2025

  75. [84]

    The shaky foundations of large lan- guage models and foundation models for electronic health records

    Michael Wornow, Yizhe Xu, Rahul Thapa, Birju Patel, Ethan Steinberg, Scott Fleming, Michael A Pfeffer, Jason Fries, and Nigam H Shah. The shaky foundations of large lan- guage models and foundation models for electronic health records. npj Digital Medicine, 6(1): 135, 2023

    2023

  76. [85]

    Context clues: Evaluating long context models for clinical prediction tasks on ehr data

    Michael Wornow, Suhana Bedi, Miguel Angel Fuentes Hernandez, Ethan Steinberg, Jason Alan Fries, Christopher Re, Sanmi Koyejo, and Nigam Shah. Context clues: Evaluating long context models for clinical prediction tasks on ehr data. In The Thirteenth International Conference on ...

    2024

  77. [86]

    Ehrshot: An ehr benchmark for few-shot evaluation of foundation models

    Michael Wornow, Rahul Thapa, Ethan Steinberg, Jason Fries, and Nigam Shah. Ehrshot: An ehr benchmark for few-shot evaluation of foundation models. Advances in Neural Information Processing Systems, 36, 2024

    2024

  78. [87]

    McDermott

    Justin Xu, Jack Gallifant, Alistair Johnson, and Matthew B.A. McDermott. ACES: Automatic co- hort extraction system for event-stream datasets. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=P4XmKjXTrM

    2025

  79. [88]

    A large language model for electronic health records

    Xi Yang, Aokun Chen, Nima PourNejatian, Hoo Chang Shin, Kaleb E Smith, Christopher Parisien, Colin Compas, Cheryl Martin, Anthony B Costa, Mona G Flores, et al. A large language model for electronic health records. NPJ digital medicine, 5(1):194, 2022. 16

    2022

  80. [89]

    at-risk cohort inclusion event

    Zhichao Yang, Avijit Mitra, Weisong Liu, Dan Berlowitz, and Hong Yu. Transformehr: transformer-based encoder-decoder generative model to enhance prediction of disease out- comes using electronic health records. Nature communications, 14(1):7857, 2023. 17 A Existing EHR foundat...

    2023

  81. [2021]

    URL https://doi.org/10.1146/ annurev-biodatasci-092820-114757

    doi: 10.1146/annurev-biodatasci-092820-114757. URL https://doi.org/10.1146/ annurev-biodatasci-092820-114757

    doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.