Pith. sign in

REVIEW 4 cited by

ClinicalMamba: A Generative Clinical Language Model on Longitudinal Clinical Notes

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2403.05795 v1 pith:M3MHABGY submitted 2024-03-09 cs.CL

Zhichao Yang , Avijit Mitra , Sunjae Kwon , Hong Yu This is my paper · ORCID
classification cs.CL
keywords clinicallanguageclinicalmambainformationnoteslongitudinalmodelmodels
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

The advancement of natural language processing (NLP) systems in healthcare hinges on language model ability to interpret the intricate information contained within clinical notes. This process often requires integrating information from various time points in a patient's medical history. However, most earlier clinical language models were pretrained with a context length limited to roughly one clinical document. In this study, We introduce ClinicalMamba, a specialized version of the Mamba language model, pretrained on a vast corpus of longitudinal clinical notes to address the unique linguistic characteristics and information processing needs of the medical domain. ClinicalMamba, with 130 million and 2.8 billion parameters, demonstrates a superior performance in modeling clinical language across extended text lengths compared to Mamba and clinical Llama. With few-shot learning, ClinicalMamba achieves notable benchmarks in speed and accuracy, outperforming existing clinical language models and general domain large models like GPT-4 in longitudinal clinical notes information extraction tasks.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Clinical trial cohort selection using Large Language Models on n2c2 Challenges

    cs.CL 2025-01 conditional novelty 5.0 of 10

    Open-source LLMs achieve moderate F1 on straightforward clinical trial criteria but underperform challenge-winning systems on criteria requiring fine-grained reasoning across three n2c2 datasets.

  2. MASV: Speaker Verification with Global and Local Context Mamba

    eess.AS 2024-12 conditional novelty 5.0 of 10

    MASV is an ECAPA-TDNN variant whose Res2 blocks are replaced by local-context and tri-directional Mamba blocks, improving speaker verification EER and lowering FLOPS on a private dataset.

  3. Rethinking the long-range dependency in Mamba/SSM and transformer models

    cs.LG 2025-09 reject novelty 3.0 of 10

    SSM/Mamba long-range dependency decays exponentially with the time gap by construction; a proposed interaction-based hidden state update can break this decay, but its proven stability covers only a restrictive special case.

  4. The Rise of Small Language Models in Healthcare: A Comprehensive Survey

    cs.CL 2025-04 conditional novelty 3.0 of 10

    A comprehensive survey of small language models in healthcare, with a taxonomy of building, adapting, and compressing them for clinical NLP tasks.

Pith tools