Temporal Entailment Pretraining for Clinical Language Models over EHR Data

Fi Zheng; Kai Sato; Shi Li; Tatsunori Tanaka; Yuanyun Zhang; ZhiFeng Li

REVIEW 5 major objections 6 minor 14 references

Temporal Entailment Pretraining for Clinical Language Models over EHR Data

T0 review · 5 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Temporal Entailment Pretraining teaches clinical language models to decide whether a later EHR state is entailed, contradicted, or neutral with respect to an earlier state, improving temporal clinical tasks.

desk verdict TEP is a plausible new pretraining objective, but the empirical claims need stronger validation—patient leakage and circular evaluation are unresolved. read the letter →

arxiv 2504.18128 v1 pith:NEJ3GVRU submitted 2025-04-25 cs.CL cs.LG

Tatsunori Tanaka , Fi Zheng , Kai Sato , Zhifeng Li , Yuanyun Zhang , Shi Li This is my paper · ORCID

classification cs.CLcs.LG

keywords temporalentailmentclinicallanguagemodelsEHRpretrainingweaksupervisionreasoningdiseaseprogressionearlywarningpredictionnaturalinference

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that temporality should be a first-class pretraining signal for clinical language models, not an afterthought. It introduces Temporal Entailment Pretraining (TEP), which turns a patient's EHR timeline into ordered sentence pairs and trains the model to classify the later state as entailed by, contradictory to, or neutral with respect to the earlier state. Using about 3.2 million weakly labeled pairs from MIMIC-IV, TEP improves performance over strong baselines on temporal clinical question answering, early warning prediction, and disease progression modeling. If the claim is right, clinical models can acquire temporal reasoning at scale without manual annotation, by learning from the structure already present in electronic health records.

What carries the argument

The central object is the temporal entailment pair $(x_t, x_{t'})$ with $t<t'$, drawn from the same patient timeline and labeled $y\in\{\text{entail},\text{contradict},\text{neutral}\}$ by weak-supervision heuristics. The model is a transformer encoder with rotary positional embeddings, fed as `[CLS] $x_t$ [SEP] $x_{t'}$`; the CLS vector goes through a classification head trained with a cross-entropy loss (with label smoothing) over the three relations. An optional soft-margin order loss pushes entailed pairs toward a partial order in embedding space, $x_t \preceq x_{t'}$, so that vector differences encode directional clinical progression, while contradictions diverge and neutral pairs stay far apart. This machinery is what carries the argument: the classification objective creates the temporal supervision, RoPE encodes relative time offsets, and the order geometry gives the learned space a directional structure that transfers to forecasting and diagnosis tasks.

What would settle it

One can settle this by pretraining TEP on a corpus from which all pairs whose labels were generated by the same progression rules as the evaluation target (for example, CKD staging rules) are removed, then re-running the disease-progression evaluation. If the reported macro-F1 and MCC gains collapse, the gains are attributable to memorizing those labeling rules rather than to general temporal reasoning.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that a temporal version of natural language inference is a learnable and transferable pretraining objective for clinical data. Given two temporally separated segments from the same patient, the model predicts whether the later segment is entailed, contradicted, or neutral relative to the earlier one; the authors show that this objective, trained with weak supervision from diagnostic ontologies and progression patterns, yields a model that outperforms all baselines on every evaluation task. Concretely, TEP reaches 81.4 QA F1 versus 71.2 for ClinicalBERT, 85.9 early-warning AUROC versus 78.5, and 73.6 CKD macro-F1 versus 64.3, with a 0.67 MCC compared with 0.51. The authors interpret these results as evidence that clinical language models should be pretrained to reason over time rather than merely to reconstruct masked tokens.

Load-bearing premise

The load-bearing premise is that the weak-supervision heuristics assign correct entailment labels—particularly the diagnostic progression rules such as CKD stage 2 entailing stage 4—so that the model learns general temporal reasoning instead of memorizing a fixed set of labeling rules that also define the evaluation target.

Editorial extensions

If this is right

TEP retains 89.2% of its full-data QA performance when fine-tuned on only 10% of the data, compared with 72.8% for ModernBERT, so temporal pretraining appears to reduce the amount of labeled fine-tuning data needed.
Calibration remains stable across time gaps up to 30 days, whereas MLM-based models degrade beyond 10 days, so predictions about distant future states remain well calibrated.
Removing RoPE or weakly labeled contradictions each reduce QA F1 by more than 3 points, so both the temporal positional encoding and the contradiction signal are load-bearing components.
TEP outperforms Retrospective T5, a generative future-segment completion model, on all three evaluation tasks, suggesting classification-style temporal supervision transfers better than autoregressive completion in this setup.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A decisive test of the mechanism is to pretrain TEP with the disease-progression heuristics removed and evaluate on independently annotated temporal benchmarks; if gains survive, the learned signal is general temporal reasoning rather than rule memorization.
Because the input representation is text-based and modality-agnostic, the same objective could be applied to structured event streams, imaging reports, or mixed-modal EHRs, potentially unifying currently separate text-based and structured-data foundation models.
The order-embedding geometry suggests that trajectory-level vector arithmetic could support counterfactual-style queries, such as subtracting a baseline state from a worsening state to identify which clinical features drive progression; the paper does not test this, so it remains an open possibility.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The new thing here is real: framing clinical pretraining as temporal entailment between time-separated EHR segments is a sensible idea, and the authors execute it with a reasonable pipeline (weak supervision, RoPE, label smoothing). That part is worth taking seriously. I don't see this exact objective in the cited literature, and the writing is clear.

The soft spots are substantial. First, there is no patient-level split between pretraining and downstream evaluation. They pretrain on 500K MIMIC-IV patients and then fine-tune/evaluate on MIMIC-IV cohorts without saying those sets are disjoint. If the same patients' records appear in both stages, the model can memorize idiosyncratic trajectories, and the reported gains (8.3% QA F1, etc.) become uninformative. The temporal-gap subsampling in Section 3.8 prevents intra-patient leakage, but not cross-patient leakage.

Second, the disease progression evaluation is close to circular. Section 3.3 builds 'entail' labels from progression rules like 'stage 2 CKD to stage 4 CKD', and Section 4.3 asks models to predict future CKD stages from initial diagnosis. A model that memorizes those rules will score well without learning generalizable temporal reasoning. The authors only mention weak-label noise generically, not this specific contamination.

Third, the SOTA claim lacks statistical support: no error bars, no significance tests, and no comparison to CEHR-BERT or MOTOR, even though both are cited. That alone would force a conditional verdict.

There are also signs of carelessness that make me cautious: unresolved citation placeholders ('[?]'), a fake grant number (R01EB999999), and a code/data link that appears to be a stub. None of these kill the idea, but they suggest the manuscript wasn't fully vetted.

If the authors add a patient-level split, rework the CKD evaluation to avoid label leakage, and include error bars plus the missing baselines, the paper could be solid. Right now it's a promising idea with unsubstantiated results. I'd send it to peer review because the core contribution deserves scrutiny, but I would not cite it in my own work until the empirical claims hold up. It's a good reading-group case study on leakage and circularity in clinical ML evaluation.

Referee Report

5 major / 6 minor

Summary. The paper introduces Temporal Entailment Pretraining (TEP), a self-supervised objective in which a clinical language model is trained to classify whether a later EHR segment is entailed by, contradicts, or is neutral with respect to an earlier segment from the same patient. Training pairs are built from MIMIC-IV records and labeled with weak heuristics derived from diagnostic ontologies, lab-value changes, and medication sequences. The authors pretrain a ModernBERT encoder with RoPE on 500K patients and 3.2M pairs, then evaluate on temporal clinical QA, early warning prediction, and CKD disease progression, reporting large gains over ClinicalBERT, ModernBERT, ClinicalBERT+BiLSTM, and Retrospective T5 in Table 1. The paper also presents ablations on temporal calibration, label density, and zero-shot QA, and argues that TEP induces a partially ordered representation space. The central claim is that TEP provides a scalable pretraining objective that improves generalizable temporal reasoning in clinical language models.

Significance. If the empirical claims are validated, TEP would be a useful and scalable pretraining objective for clinical NLP: it is self-supervised, does not require manual annotation, and directly targets a temporal dimension that masked-language-model pretraining ignores. The idea of casting clinical trajectory modeling as temporal NLI with weak supervision is plausible and could transfer to other EHR tasks. However, the significance assessment currently rests on a small set of empirical results that have several unaddressed validity threats, so the contribution is best viewed as an interesting proposal whose demonstrated state-of-the-art status is not yet established.

major comments (5)

[Sections 4.1 and 4.3] No patient-level split between pretraining and downstream evaluation is described. Pretraining uses over 500,000 MIMIC-IV patients, and the early warning and CKD progression tasks use MIMIC-IV cohorts; if the same patients appear in both stages, the model can memorize idiosyncratic trajectory patterns rather than learn transferable temporal reasoning, which would inflate all three metrics in Table 1. The authors should state explicitly whether pretraining and evaluation patient sets are disjoint, and if they are not, re-run the experiments with a strict patient-level split.
[Table 1] The main results are reported as single numbers with no variance, no multiple seeds, and no significance tests. An 8.3-point QA F1 gain over ClinicalBERT could be within run-to-run noise, especially given the coarse hyperparameter search grid described in Section 4.2. The paper should report means and standard deviations over at least three seeds, along with appropriate significance testing or confidence intervals, before claiming state-of-the-art performance.
[Sections 3.3 and 4.3] The CKD disease progression evaluation is partially circular. Section 3.3 constructs entailment labels from diagnostic progression ontologies (including CKD stage progression), and Section 4.3 evaluates the model on predicting future CKD stages from an initial diagnosis. A model that memorizes the same progression rules used for pretraining can score artificially high on this task without learning generalizable temporal reasoning. The authors should either evaluate on a progression task whose labels were not used in pretraining, use an external benchmark such as EHR-shot or MEDS-DEV, or explicitly measure how much of the CKD gain comes from rule memorization versus generalization.
[Section 3.3] The quality of the weak labels is never measured. The paper relies on heuristics such as ICD ontology progression, lab quantile changes, and synthetic contradictions, but reports no manual validation, no agreement with MedNLI-style annotations, and no per-heuristic accuracy. Since the pretraining signal depends entirely on these labels, the authors should provide a validation study of weak-label quality and report ablation results that show each heuristic's contribution to downstream performance.
[Sections 2.3 and 4.2] CEHR-BERT and MOTOR are cited as relevant temporal EHR foundation models but are not included as baselines in Table 1. Given that the paper claims to outperform all baselines and to achieve state-of-the-art results, the absence of these comparators weakens the claim, especially because they were designed specifically for temporal EHR modeling. The authors should either include these models in the comparison or explicitly state the reasons for omission and qualify the state-of-the-art claim accordingly.

minor comments (6)

[Sections 1 and 3.6] There are unresolved citation placeholders: '[?]' appears for SNLI/MNLI in the introduction and for order embeddings in Section 3.6 and Appendix A. These references need to be completed.
[Section 4.3] The temporal QA task is said to use 12k manually curated QA pairs from MIMIC-IV and Mayo Clinic data, but no details are given about the Mayo Clinic data, its licensing, collection protocol, or the split between the two sources. Without this information the evaluation is not reproducible.
[Appendix C.2] Table 3 reports improvements with pair density but no variance or significance measures, so it is unclear whether the trend is reliable.
[Section 4.5] The low-resource claim that TEP retains 89.2% of full-data QA performance versus 72.8% for ModernBERT is presented without a description of the 10% subsampling procedure, number of runs, or error bars; this claim should be reported with the same rigor as the main table.
[Throughout] There are several typos and inconsistent terms, e.g., 'Clinial Longformer' in Section 2.1, 'Conlcuding Remarks' in Section 5, and inconsistent use of 'ClinicalBERT' to refer to two different works in Sections 2.1 and 2.2. A careful proofreading pass is needed.
[Section 5 and Appendix E] The paper promises release of code and data at a GitHub URL, but no repository is provided and the URL appears to be a placeholder. If code and data generators are to be released, the manuscript should state the expected availability or provide a working link.

Circularity Check

1 steps flagged · score 6.0 of 10

Disease-progression evaluation is partially circular: the TEP weak labels and the CKD staging target come from the same diagnostic-progression ontology.

self definitional [Section 3.3 (weak labeling) vs Section 4.3 (Disease Progression Modeling), with Appendix B]
"Weak supervision ... is applied to assign y using heuristics derived from diagnostic ontologies ... For example, progression from 'stage 2 chronic kidney disease' to 'stage 4 CKD' entails a worsening trajectory (entail). ... Appendix B: Entailment labels are generated from known diagnosis progressions (e.g., from ICD ontologies), changes in lab value quantiles (e.g., GFR) ... 4.3: Models predict future disease stages (e.g., CKD stages 1-5, NYHA classification for heart failure) given an initial diagnosis and subsequent labs and notes."

The TEP pretraining labels for disease progression are generated from diagnostic/ICD ontologies, with CKD stage advancement as the canonical entailment example. The downstream Disease Progression task asks the model to predict exactly the same kind of future disease stages from an initial diagnosis. Thus the reported CKD Macro-F1/MCC can be achieved by applying the ontological progression rules already memorized during pretraining; the evaluation target is the same relation that generated the supervision. This makes the disease-progression 'prediction' partially reduce by construction. QA and early warning tasks are not built from the same label generator, so the circularity is partial rather than total.

full rationale

The clearest circular step is in the disease-progression evaluation: Section 3.3 constructs entailment labels from diagnostic ontologies and UMLS progression patterns, and Section 4.3 evaluates 'disease progression modeling' by predicting future CKD stages from initial diagnosis. A model can memorize these ontological progression rules during TEP pretraining and then 'predict' the same progression at evaluation, so the Table 1 CKD gains are partly forced by construction. The temporal clinical QA and early-warning tasks use manually curated QA pairs and deterioration events that are not derived from the same weak-label generator, so those results remain independent evidence. No load-bearing self-citation chain is present: the comparisons are to external baselines and the cited prior work (Steinberg et al., Wornow et al., Alsentzer et al.) is not by the present authors. The absence of a patient-level split and of error bars is a correctness and robustness concern, not a circularity defect, and is not scored here. Overall, one of the three headline evaluations reduces to its own training signal, giving a 6 for partial circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper does not derive any quantity from first principles; it makes an empirical claim supported by training and evaluation. The key load-bearing assumptions are the validity of weak labels, the transferability of temporal entailment, and the absence of data leakage between pretraining and evaluation. The main hand-chosen settings are the temporal gap bounds, the entailment-pair density, and the unreported label smoothing.

free parameters (4)

Temporal gap bounds delta_min and delta_max
Section 3.8 subsamples pairs with delta_min < |t' - t| < delta_max; exact values are not reported and control label difficulty.
Weak-label thresholds (e.g., lab value quantiles)
Appendix B uses changes in lab value quantiles such as GFR to assign entailment and contradiction; the thresholds are unspecified.
Entailment pair density = 10 pairs per patient
Table 3 indicates the main results correspond to 10 pairs per patient; density materially changes downstream scores.
Label smoothing weight
Section 3.5 applies label smoothing to handle weak-label noise, but the weight is not reported.

assumptions (4)

domain assumption Weak supervision heuristics correctly assign entailment labels.
Labels come from ontologies and lab-quantile rules (Section 3.3, Appendix B); no precision or recall analysis is reported, and Section 5 acknowledges label noise.
domain assumption Temporal entailment pretraining transfers to downstream clinical tasks.
The paper's central claim is that this objective improves QA, early warning, and progression; transfer is asserted from downstream results.
domain assumption Pretraining and evaluation data do not overlap in a way that inflates scores.
Only temporal leakage is addressed in Section 3.8; patient-level splits between MIMIC-IV pretraining and MIMIC-IV evaluation are not described.
domain assumption Manual curation of the QA pairs is accurate and unbiased.
Section 4.3 treats 12k manually curated QA pairs as gold; no inter-annotator agreement or curation protocol is given.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Temporal Entailment Pretraining for Clinical Language Models over EHR Data." pith.science (2026). https://pith.science/paper/NEJ3GVRU

@misc{pith2026250418128,
  author       = {Pith},
  title        = {Pith review of: Temporal Entailment Pretraining for Clinical Language Models over EHR Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NEJ3GVRU}},
  note         = {Machine review of arXiv:2504.18128}
}

read the original abstract

Clinical language models have achieved strong performance on downstream tasks by pretraining on domain specific corpora such as discharge summaries and medical notes. However, most approaches treat the electronic health record as a static document, neglecting the temporally-evolving and causally entwined nature of patient trajectories. In this paper, we introduce a novel temporal entailment pretraining objective for language models in the clinical domain. Our method formulates EHR segments as temporally ordered sentence pairs and trains the model to determine whether a later state is entailed by, contradictory to, or neutral with respect to an earlier state. Through this temporally structured pretraining task, models learn to perform latent clinical reasoning over time, improving their ability to generalize across forecasting and diagnosis tasks. We pretrain on a large corpus derived from MIMIC IV and demonstrate state of the art results on temporal clinical QA, early warning prediction, and disease progression modeling.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

14 extracted references · 3 canonical work pages

[7]

Enhancing Antibiotic Stewardship using a Natural Language Approach for Better Feature Representation

Simon A Lee, Trevor Brokowski, and Jeffrey N Chiang. Enhanci ng antibiotic stewardship using a natural language approach for better feature representation. arXiv preprint arXiv:2405.20419 , 2024a. Simon A Lee, Sujay Jain, Alex Chen, Kyoka Ono, Jennifer Fang, Akos Rudas, and Jeffrey N Chiang. Emergency department decision support using clinical pseudo-note...

work page Pith review arXiv
[9]

Mostafa Dehghani, Hamed Zamani, Aliaksei Severyn, Jaap Kam ps, and W Bruce Croft

URL https://arxiv.org/abs/1904.03323. Mostafa Dehghani, Hamed Zamani, Aliaksei Severyn, Jaap Kam ps, and W Bruce Croft. Neural ranking models with weak supervision. In Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval, pages 65–74,

arXiv 1904
[10]

Text serialization and their relat ionship with the conventional paradigms of tabular machine learning

Kyoka Ono and Simon A Lee. Text serialization and their relat ionship with the conventional paradigms of tabular machine learning. arXiv preprint arXiv:2406.13846 ,

arXiv
[11]

Lessons from natural language inference in the clinical domain

Alexey Romanov and Chaitanya Shivade. Lessons from natural language inference in the clinical domain. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’i chi Tsujii, editors, Proceedings of the 2018 Conference on Empirical Methods in Natural Language Proces sing, pages 1586–1596, Brussels, Belgium, October-November

work page 2018
[12]

doi: 10.18653/v1/D18-1187

Association for Computational Lin guistics. doi: 10.18653/v1/D18-1187. URL https://aclanthology.org/D18-1187/. Julian Salazar, Davis Liang, Toan Q Nguyen, and Katrin Kirch hoff. Masked language model scoring. arXiv preprint arXiv:1910.14659,

arXiv
[13]

Jianlin Su, Y u Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Y unfeng Liu

URL https://arxiv.org/abs/2301.03150. Jianlin Su, Y u Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Y unfeng Liu. Roformer: Enhanced transformer with rotary position embedding,

arXiv
[14]

URL https://arxiv.org/abs/2104.09864. Benjamin Warner, Antoine Chafﬁn, Benjamin Clavi´ e, Orion Weller, Oskar Hallstr¨ om, Said Taghadouini, Alexis Gal- lagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, Nathan Coop er, Grifﬁn Adams, Jeremy Howard, and Iacopo Poli. Smarter, better, faster, longer: A modern bidirectio nal encoder for fast, memory efﬁcient, an...

arXiv
[15]

Michael Wornow, Rahul Thapa, Ethan Steinberg, Jason Fries, and Nigam Shah

URL https://arxiv.org/abs/2412.13663. Michael Wornow, Rahul Thapa, Ethan Steinberg, Jason Fries, and Nigam Shah. Ehrshot: An ehr benchmark for few-shot evaluation of foundation models. Advances in Neural Information Processing Systems , 36:67125–67137, 2023a. Michael Wornow, Yizhe Xu, Rahul Thapa, Birju Patel, Ethan Steinberg, Scott Fleming, Michael A Pfe...

arXiv

Show all 14 references

[2018]

Deep learning based question generation using t5 transformer

Khushnuma Grover, Katinder Kaur, Kartikey Tiwari, Rupali, and Parteek Kumar. Deep learning based question generation using t5 transformer. In Advanced Computing: 10th International Conference, IACC 2 020, Panaji, Goa, India, December 5–6, 2020, Revised Selected Papers, Part I ...

2020
[2019]

Rodney A Gabriel, Tsung-Ting Kuo, Julian McAuley, and Chun- Nan Hsu

URL https://arxiv.org/abs/1810.04805. Rodney A Gabriel, Tsung-Ting Kuo, Julian McAuley, and Chun- Nan Hsu. Identifying and characterizing highly similar notes in big clinical note datasets. Journal of biomedical informatics , 82:63–69,

arXiv
[2021]

doi: 10.18653/v1/2021.acl-short.129

Association for Computational Linguist ics. doi: 10.18653/v1/2021.acl-short.129. URL https://aclanthology.org/2021.acl-short.129/. Jinmiao Huang, Cesar Osorio, and Luke Wicent Sy. An empirical evaluation of deep learning for icd-9 code assignment using mimic-iii clinical notes...

2021 arXiv
[2022]

Ning Liu, Qian Hu, Huayun Xu, Xing Xu, and Mengxin Chen

U RL https://arxiv.org/abs/2201.11838. Ning Liu, Qian Hu, Huayun Xu, Xing Xu, and Mengxin Chen. Med-b ert: A pretraining framework for medical records named entity recognition. IEEE Transactions on Industrial Informatics , 18(8):5600–5608,

arXiv
[2023]

Christine Herlihy and Rachel Rudinger

URL https://arxiv.org/abs/2210.10723. Christine Herlihy and Rachel Rudinger. MedNLI is not immune : Natural language inference artifacts in the clinical domain. In Chengqing Zong, Fei Xia, Wenjie Li, a nd Roberto Navigli, editors, Proceed- ings of the 59th Annual Meeting of th...

arXiv
[2024]

Can large language models ab stract medical coded language? arXiv preprint arXiv:2403.10822,

Simon A Lee and Timothy Lindsey. Can large language models ab stract medical coded language? arXiv preprint arXiv:2403.10822,

arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [7]

Enhancing Antibiotic Stewardship using a Natural Language Approach for Better Feature Representation

Simon A Lee, Trevor Brokowski, and Jeffrey N Chiang. Enhanci ng antibiotic stewardship using a natural language approach for better feature representation. arXiv preprint arXiv:2405.20419 , 2024a. Simon A Lee, Sujay Jain, Alex Chen, Kyoka Ono, Jennifer Fang, Akos Rudas, and Jeffrey N Chiang. Emergency department decision support using clinical pseudo-note...

work page Pith review arXiv

[2] [9]

Mostafa Dehghani, Hamed Zamani, Aliaksei Severyn, Jaap Kam ps, and W Bruce Croft

URL https://arxiv.org/abs/1904.03323. Mostafa Dehghani, Hamed Zamani, Aliaksei Severyn, Jaap Kam ps, and W Bruce Croft. Neural ranking models with weak supervision. In Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval, pages 65–74,

arXiv 1904

[3] [10]

Text serialization and their relat ionship with the conventional paradigms of tabular machine learning

Kyoka Ono and Simon A Lee. Text serialization and their relat ionship with the conventional paradigms of tabular machine learning. arXiv preprint arXiv:2406.13846 ,

arXiv

[4] [11]

Lessons from natural language inference in the clinical domain

Alexey Romanov and Chaitanya Shivade. Lessons from natural language inference in the clinical domain. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’i chi Tsujii, editors, Proceedings of the 2018 Conference on Empirical Methods in Natural Language Proces sing, pages 1586–1596, Brussels, Belgium, October-November

work page 2018

[5] [12]

doi: 10.18653/v1/D18-1187

Association for Computational Lin guistics. doi: 10.18653/v1/D18-1187. URL https://aclanthology.org/D18-1187/. Julian Salazar, Davis Liang, Toan Q Nguyen, and Katrin Kirch hoff. Masked language model scoring. arXiv preprint arXiv:1910.14659,

arXiv

[6] [13]

Jianlin Su, Y u Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Y unfeng Liu

URL https://arxiv.org/abs/2301.03150. Jianlin Su, Y u Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Y unfeng Liu. Roformer: Enhanced transformer with rotary position embedding,

arXiv

[7] [14]

URL https://arxiv.org/abs/2104.09864. Benjamin Warner, Antoine Chafﬁn, Benjamin Clavi´ e, Orion Weller, Oskar Hallstr¨ om, Said Taghadouini, Alexis Gal- lagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, Nathan Coop er, Grifﬁn Adams, Jeremy Howard, and Iacopo Poli. Smarter, better, faster, longer: A modern bidirectio nal encoder for fast, memory efﬁcient, an...

arXiv

[8] [15]

Michael Wornow, Rahul Thapa, Ethan Steinberg, Jason Fries, and Nigam Shah

URL https://arxiv.org/abs/2412.13663. Michael Wornow, Rahul Thapa, Ethan Steinberg, Jason Fries, and Nigam Shah. Ehrshot: An ehr benchmark for few-shot evaluation of foundation models. Advances in Neural Information Processing Systems , 36:67125–67137, 2023a. Michael Wornow, Yizhe Xu, Rahul Thapa, Birju Patel, Ethan Steinberg, Scott Fleming, Michael A Pfe...

arXiv