Toward Total Recall: Enhancing FAIRness through AI-Driven Metadata Standardization

Mark A Musen; Rafael S. Gon\c{c}alves; Sowmya S Sundaram

REVIEW 3 major objections 5 minor 1 cited by

Toward Total Recall: Enhancing FAIRness through AI-Driven Metadata Standardization

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Guiding GPT-4 with structured CEDAR metadata templates raises exact-match dataset recall from 17.65% to 62.87% in BioSample and GEO searches.

desk verdict Plausible method, but the recall gain is measured against a gold standard that counts 'lung cancer' as tissue:lung, so the headline number may be an artifact of the evaluation design. read the letter →

arxiv 2504.05307 v2 pith:PVRIVQGB submitted 2025-02-13 cs.IR cs.AI

Sowmya S Sundaram , Rafael S. Gonçalves , Mark A Musen This is my paper · ORCID

classification cs.IRcs.AI

keywords metadatastandardizationFAIRdataprincipleslargelanguagemodelsCEDARtemplatesinformationretrievalbiomedicaldiscoveryGPT-4quality

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that automatic standardization of biomedical metadata can be dramatically improved by giving a large language model structured, ontology-aligned templates rather than only a data dictionary. Using records from BioSample and GEO, the authors prompt GPT-4 to rewrite metadata according to CEDAR templates, which specify expected fields, value types, and permissible values. They report that exact-match retrieval of datasets on tissue queries rises from 17.65% average recall with the original raw metadata to 62.87% with the template-guided GPT-4 corrections, with data-dictionary-only guidance reaching about 38%. The result matters because poor metadata is a recognized barrier to dataset discovery and reuse, and the paper provides a quantitative test of the FAIR claim that structured metadata improves findability.

What carries the argument

The load-bearing mechanism is template-augmented prompt engineering built on CEDAR templates: machine-readable metadata schemas that list expected fields and restrict values, for instance requiring the tissue field to take terms from the Uberon anatomy ontology. In the GPT-4+CEDAR condition, the model is given the raw record plus a CEDAR template and asked to rewrite the record into the template's format. The template's value restrictions allow the model to recognize that a phrase such as 'lung cancer' cannot be a tissue value and to move it to the disease field, whereas a data dictionary that merely describes fields leaves more room for the model to invent or tolerate nonstandard values. The corrected metadata is then searched with exact string matching on the tissue field, so the template's role is to make values uniform enough that simple keyword queries match.

What would settle it

Manually label the true tissue type for a random sample of the 4,800 BioSample and GEO records, independent of the paper's keyword rules, then compare retrieval of baseline, GPT-4+DD, and GPT-4+CEDAR metadata under the same exact-match queries; if the human-labeled recall for GPT-4+CEDAR is not substantially above baseline, or if a synonym-aware semantic retrieval baseline reaches comparable recall without any metadata correction, the central claim would need to be revised.

Watch

Extended reading notes

Core claim

The central claim is that combining GPT-4 with CEDAR metadata templates yields substantially better dataset retrieval than either the original metadata or GPT-4 guided only by a data dictionary. On the authors' 4,800-record test set, average recall for four tissue queries rose from 17.65% at baseline to 62.87% with GPT-4+CEDAR; precision rose from 58% to 66%, and F1 rose from 24% to 63%. The improvement was consistent across the three cancer-themed cohorts and both repositories, and paired t-tests with Cohen's d greater than 0.8 marked the gains as statistically and practically meaningful. The authors also report that alternative models such as LLaMA-3 and MedLLaMA-2 produced more formatting errors and did not match GPT-4+CEDAR, and that data-dictionary guidance alone hurt GEO performance. The intended upshot is that FAIR-style metadata standardization is not only a principle but a measurable way to surface datasets that keyword searches currently miss.

Load-bearing premise

The whole comparison rests on assuming that a dataset counts as relevant to a tissue query only when its tissue field exactly contains the query word, and that the keyword rules used to build the list of correct answers are themselves right.

Editorial extensions

If this is right

Repositories can apply LLM-plus-template standardization to legacy metadata in bulk, with no retraining or architecture changes, turning records that were practically invisible to keyword search into retrievable ones.
Template-guided correction should generalize across disciplines that have community reporting guidelines and ontologies, since the gain comes from machine-actionable value constraints rather than from disease-specific tuning.
Data dictionaries alone are not enough; the comparison with GPT-4+DD implies that the value-set and ontology restrictions encoded in templates are the active ingredient.
Scaling the pipeline to the full BioSample collection, on the order of five million records, would create a large benchmark corpus of FAIR-aligned metadata and support broader AI-ready data reuse.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The reported recall numbers likely overstate the practical gain: the gold-standard labels were derived by simple keyword rules applied to the same raw metadata being corrected, so some of the 'missed' baseline records may in fact be findable by users who tolerate synonyms, partial matches, or free-text fields.
A fair comparison against modern semantic retrieval would clarify how much of the gain is due to standardization versus to the brittleness of the exact-match retrieval baseline; semantic search might narrow the gap substantially.
The template advantage suggests a testable extension: ablating the ontology value restriction while keeping field names would isolate whether ontological constraints, rather than simple structure, drive the improvement.
Because the same templates could be applied to any text metadata with a controlled vocabulary, the method could transfer to non-biomedical repositories, such as environmental or materials science, where community ontologies exist.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's my take on arXiv:2504.05307. The headline result—GPT-4+CEDAR lifting recall from ~18% to ~63% on a 4,800-record test set—is plausible as an internal measurement, but the evaluation design makes it hard to trust as evidence that metadata were actually corrected. The ground-truth labels in Table 2 are keyword rules that assign 'lung' to any tissue field containing the substring 'lung', including 'lung cancer'. Yet Figure 1 in the paper explicitly uses 'lung cancer' as an example of an incorrect tissue value. So the gold standard is counting the very error the method is supposed to fix as a true positive. That's a load-bearing circularity, not a minor quibble.

What's genuinely new: the paper offers a quantitative retrieval comparison on a reasonably sized corpus, with CEDAR templates as a guiding structure, and it tests multiple LLMs. That is useful operational evidence for repository managers. The software is public and the experiments are straightforward to reproduce. The writing is clear, and the authors acknowledge the approximate nature of the gold standard, though they understate how problematic it is.

Other soft spots: retrieval is exact-match on a single field, which is not how real search works. There's no external validation or comparison to simpler standardization baselines (e.g., a dictionary lookup). The statistical tests are paired t-tests, but with only four queries per condition, the effective sample size is tiny; the reported p-values don't mean much. The FAIR-validating claims in the discussion overreach—this is a single retrieval model on two databases, not a proof that FAIR works.

The method itself is not useless. Standardizing free text into a controlled vocabulary will generally make exact-match retrieval work better. But the paper doesn't distinguish vocabulary alignment from genuine metadata correction. The good news is the evaluation is fixable: sample a subset of records, have an expert assign true tissue labels, compare against the raw and standardized values. That would tell you whether the recall gain is real and whether the method introduces errors.

Who should read it: people working on metadata curation, repository managers, and researchers interested in LLM+knowledge-base pipelines. It deserves a serious referee, because the core question is important and the method is practical. I'd recommend conditional acceptance with a request for an independent gold standard.

That's my view.

Referee Report

3 major / 5 minor

Summary. The paper proposes an automated metadata standardization pipeline that combines GPT-4 with structured CEDAR templates, and evaluates it on 4,800 records sampled from NCBI BioSample and GEO. The authors report that exact-match retrieval on the standardized metadata, using queries such as tissue:lung, improves average recall from 17.65% with raw metadata to 62.87% with GPT-4+CEDAR, with smaller gains when GPT-4 is guided only by a data dictionary. They also compare GPT-4 with LLaMA-3 and MedLLaMA-2, and report statistically significant improvements with large effect sizes. The paper frames the result as empirical evidence that FAIR-aligned, structured metadata improves dataset findability.

Significance. If the reported effect is real, the contribution is significant: it would provide a quantitative demonstration that LLMs augmented with ontology-constrained templates can improve metadata quality and searchability at repository scale, with practical implications for FAIR data remediation. Strengths include the use of real repository data, public code and data, comparisons across multiple LLMs, and explicit reporting of recall, precision, and F1. The main weakness is that the evaluation's ground truth is constructed from the very same raw metadata field that the method is designed to correct, which risks making the headline recall improvement an artifact of vocabulary normalization rather than evidence of genuinely improved retrieval relevance.

major comments (3)

[Evaluation Metrics, Table 2] The approximate gold standard is load-bearing and is constructed from the same raw metadata field that the method is meant to correct. Table 2 assigns the label 'lung' to any tissue value containing the substring 'lung', so a record with tissue:'lung cancer' is counted as relevant to the query tissue:lung. However, Figure 1 explicitly presents 'tissue: lung cancer' as the motivating example of an inaccurate metadata value, because lung cancer is not a type of tissue. Thus the gold standard counts the paper's own example of a metadata error as a true positive for the organ query. Because the queries use the same keywords as the labeling rules, a record is relevant iff its original tissue field contains the query term, and baseline exact-match retrieval can only succeed when the raw value already equals the canonical term. The reported recall improvement therefore conflates string-to-vocabulary normalization with genuine correction. An independent gold standard, for example expert annotation of a sample's true tissue type or validation against other sample attributes, is required to support the claim that standardization improves retrieval of relevant datasets.
[Materials and Methods, Evaluation Metrics; Results] The retrieval evaluation uses exact string matching on a single metadata field, with no partial matching, stemming, synonym expansion, or use of other fields. This model is closely matched to the method's output, since GPT-4+CEDAR is prompted to produce canonical Uberon terms that equal the query strings. The paper does not compare against a non-LLM baseline that maps raw tissue strings to Uberon terms with a simple dictionary or ontology lookup. Such a baseline is necessary to attribute the observed gains to the combination of LLM and template, rather than to the normalization step itself. The authors should also report whether the improvement persists under a more realistic retrieval model, such as tokenized search over all metadata fields, which is closer to how users actually search repositories like GEO and BioSample.
[Results, statistical analysis] The statistical analysis is reported only as 'paired t-tests' with p < 0.01 and Cohen's d values, but the unit of analysis is not specified. Recall appears to be computed per query per dataset, which would yield only a handful of paired observations (at most six to eight, depending on how the four tissue queries map to the three disease cohorts). With such a small number of clusters, a paired t-test is questionable, and the reported p-values and effect sizes should be accompanied by the number of paired observations, per-query recall values, and a more appropriate analysis such as a permutation test or mixed-effects model. Without this, the claim that the improvements are statistically significant is not adequately supported.

minor comments (5)

[Materials and Methods, Dataset] Table 1 lists queries for three diseases (lung, liver, ovarian cancer), but the Results mention four tissue queries (tissue:lung, tissue:blood, tissue:liver, tissue:ovary). The paper should clarify how these four queries are assigned to the three cohorts, and whether the blood query is applied to all cohorts or only a subset.
[Results, Figures 5-7] The text refers to precision as 'Figure 7' and F1-score as 'Figure 8', but only Figures 5, 6, and 7 are present in the manuscript, and their captions indicate Figure 5 is recall, Figure 6 is precision, and Figure 7 is F1. The in-text references should be corrected.
[Materials and Methods, Approach] For GEO records, the paper reuses the BioSample data dictionary and CEDAR template because GEO lacks analogous resources. This is a reasonable approximation, but it should be explicitly listed as a limitation in the Discussion, since it may contribute to the smaller recall gains observed for GEO and affects the interpretation of the cross-repository comparison.
[Materials and Methods, Approach] There is a typo in the sentence 'so resused relevant sections of the BioSample CEDAR template'; it should read 'reused'.
[Results] The headline numbers (17.65% baseline recall to 62.87% GPT-4+CEDAR recall) are reported as averages, but the paper does not provide a per-query table with recall, precision, and F1 values, nor confidence intervals. Such a table would make the results more reproducible and allow readers to assess variability across queries and datasets.

Circularity Check

2 steps flagged · score 6.0 of 10

Evaluation is a closed loop: the approximate gold standard labels a record 'lung' iff the raw tissue field contains 'lung', and retrieval is exact match to 'lung', so the reported recall gain measures keyword alignment rather than corrected metadata.

self definitional [Materials and methods, Evaluation Metrics (Table 2) and Results; see also Introduction (Fig. 1)]
"Because no formal gold standard was available, we manually examined the “tissue” values in our test dataset and developed simple rules to assign the correct tissue values (Table 2). ... If tissue field contains the word “lung”, set label to “lung” ... The search was based on exact string matching between the standardized metadata field values and the query terms. Specifically, a metadata record was counted as retrieved if the value in its “tissue” field matched the query string exactly, without partial matching, stemming, or synonym expansion."

The ground truth is constructed from the same raw metadata that the method standardizes: 'correct' tissue labels are assigned by substring rules (e.g., any value containing 'lung' is labeled 'lung'), and relevance for a tissue:lung query is therefore true iff the raw value contains 'lung'. Retrieval is then defined as exact string match of the standardized value to 'lung'. The paper's own motivating example (Fig. 1) asserts that 'lung cancer' is not a valid tissue value, yet Table 2 counts it as relevant to tissue:lung. Consequently, the baseline misses 'lung cancer' because exact match fails, while GPT-4+CEDAR, constrained to Uberon values, outputs 'lung' and is counted as retrieved.
fitted input called prediction [Materials and methods, Evaluation Metrics: 'we manually examined the “tissue” values in our test dataset and developed simple rules to assign the correct tissue values (Table 2)']
"we manually examined the “tissue” values in our test dataset and developed simple rules to assign the correct tissue values (Table 2). For example, ‘blood’ is a commonly found tissue value, accounting for almost 50% of samples from BioSample. However, the metadata value for blood is often confounded by the methods of sample preparation, such as PBMC, whole blood, blood sample, plasma etc. Hence, we used a heuristic approach (Table 2) to assign the correct field value."

The gold-standard annotation rules are derived after inspecting the very test records that are later scored. The rules are not an external or independently validated standard; they are the authors' re-description of the raw values in the test set. The retrieval evaluation then 'predicts' whether GPT-4 reproduces those same keyword categories. This is analogous to fitting a labeling function to the test data and then reporting recall against that fitted function: the metric is forced by the way the labels were generated, and any model that mechanically emits the matched keyword will be scored as correct even when the underlying metadata correction is wrong (e.g., moving 'lung cancer' to the disease field is not distinguished from leaving it in the tissue field as 'lung').

full rationale

The strongest independent content is the engineering claim that feeding CEDAR templates to GPT-4 produces tissue values that exactly match simple query strings more often than raw metadata does; that may be useful for search normalization. But the paper's headline recall numbers are not an independent validation of metadata correctness because the relevance labels (Table 2) are substring rules over the test metadata itself, the queries use the same keywords, and retrieval is exact string match. The paper's own Figure 1 defines 'lung cancer' as an inaccurate tissue value, while Table 2 labels it as tissue 'lung'; hence the gold standard systematically counts the paper's canonical error as a true positive. This makes the reported 17.65% to 62.87% recall improvement a self-definitional artifact of the evaluation setup rather than evidence that standardized metadata is semantically correct. No other load-bearing circularity was found: the CEDAR-template and LLM components are described concretely, and the comparisons among LLMs are not circular in the same way. The method itself has content independent of the flawed evaluation, but the central quantitative claim reduces by construction to the gold-standard construction, so the score is 6.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the heuristic gold standard, the exact-match retrieval model, and the correctness of CEDAR templates. These are not externally benchmarked, so the measured recall improvements are conditional on assumptions that favor the proposed method.

free parameters (3)

Gold-standard keyword rules (Table 2) = keywords: lung, liver/HCC, ovary/ovarian, PBMC/blood, plasma, lymph; priority order as listed
These rules define relevance for computing recall and precision. They were developed after manually examining the test data, making the evaluation metric dependent on choices that favor keyword extraction.
Retrieval query set = tissue:lung, tissue:blood, tissue:liver, tissue:ovary
The four queries were hand-selected; the inclusion of blood and the exact-match syntax influence the reported aggregate recall and the baseline deficit.
Test corpus size per query = 800 records per query per database (4,800 total)
Records were sampled from 1,000 and filtered for XML errors; the selection threshold is arbitrary and affects the statistical power and generalizability.

assumptions (5)

domain assumption Relevance to a tissue query is determined by keyword presence in the original 'tissue' field.
This underlies the gold standard in Table 2; it assumes no other metadata fields are relevant and that keyword presence equals true relevance.
domain assumption Exact string matching is the appropriate retrieval model for evaluating metadata searchability.
The paper explicitly uses exact matching without stemming or synonyms, which makes free-text metadata fail query matching and inflates the benefit of standardization.
domain assumption CEDAR templates and the BioSample data dictionary are correct representations of metadata standards.
The method relies on these as ground truth for correct field values; no independent verification of their correctness is provided.
ad hoc to paper For GEO records, reusing the BioSample data dictionary and CEDAR template sections is a valid approximation.
GEO lacks its own detailed dictionary or template, so the authors reuse BioSample content; the validity of this transfer is assumed.
standard math Paired t-tests are appropriate for comparing recall values across conditions.
The statistical method is standard, but the unit of analysis (queries, datasets, or records) is not specified, which could affect the validity of the p-values.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Toward Total Recall: Enhancing FAIRness through AI-Driven Metadata Standardization." pith.science (2026). https://pith.science/paper/PVRIVQGB

@misc{pith2026250405307,
  author       = {Pith},
  title        = {Pith review of: Toward Total Recall: Enhancing FAIRness through AI-Driven Metadata Standardization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PVRIVQGB}},
  note         = {Machine review of arXiv:2504.05307}
}

read the original abstract

Scientific metadata often suffer from incompleteness, inconsistency, and formatting errors, which hinder effective discovery and reuse of the associated datasets. We present a method that combines GPT-4 with structured metadata templates from the CEDAR knowledge base to automatically standardize metadata and to ensure compliance with established standards. A CEDAR template specifies the expected fields of a metadata submission and their permissible values. Our standardization process involves using CEDAR templates to guide GPT-4 in accurately correcting and refining metadata entries in bulk, resulting in significant improvements in metadata retrieval performance, especially in recall -- the proportion of relevant datasets retrieved from the total relevant datasets available. Using the BioSample and GEO repositories maintained by the National Center for Biotechnology Information (NCBI), we demonstrate that retrieval of datasets whose metadata are altered by GPT-4 when provided with CEDAR templates (GPT-4+CEDAR) is substantially better than retrieval of datasets whose metadata are in their original state and that of datasets whose metadata are altered using GPT-4 with only data-dictionary guidance (GPT-4+DD). The average recall increases dramatically, from 17.65\% with baseline raw metadata to 62.87\% with GPT-4+CEDAR. Furthermore, we evaluate the robustness of our approach by comparing GPT-4 against other large language models, including LLaMA-3 and MedLLaMA2, demonstrating consistent performance advantages for GPT-4+CEDAR. These results underscore the transformative potential of combining advanced language models with symbolic models of standardized metadata structures for more effective and reliable data retrieval, thus accelerating scientific discoveries and data-driven research.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Knowledge engineering for open science: Building and deploying knowledge bases for metadata standards
cs.DL 2025-07 conditional novelty 2.0 of 10

CEDAR encodes community metadata standards as reusable templates that drive web forms, spreadsheets, and validators for creating FAIR research data.

Reference graph

Works this paper leans on

21 extracted references · 16 canonical work pages · cited by 1 Pith paper

[1]

Scientific data 3(1), 1–9 (2016)

Wilkinson, M.D., Dumontier, M., Aalbersberg, I.J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., Silva Santos, L.B., Bourne, P.E., et al.: The FAIR Guiding Principles for Scientific Data Management and Stewardship. Scientific data 3(1), 1–9 (2016)

work page 2016
[2]

Nature 609(7926), 222–222 (2022)

Musen, M.A.: Without Appropriate Metadata, Data-Sharing Mandates are pointless. Nature 609(7926), 222–222 (2022)

work page 2022
[3]

The variable quality of metadata about biological samples used in biomedical experiments

Gonçalves, R., Musen, M. The variable quality of metadata about biological samples used in biomedical experiments. Sci Data 6, 190021 (2019)

work page 2019
[4]

& Musen, M.A

Miron, L., Gonçalves, R.S. & Musen, M.A. Obstacles to the reuse of study metadata in ClinicalTrials.gov. Sci Data 7, 443 (2020). https://doi.org/10.1038/s41597-020-00780-z

work page doi:10.1038/s41597-020-00780-z 2020
[5]

arXiv preprint arXiv:2303.08774 (2023)

Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: GPT-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

arXiv 2023
[6]

BioProject and BioSample databases at NCBI: facilitating capture and organization of metadata

Barrett T, Clark K, Gevorgyan R, Gorelenkov V , Gribov E, Karsch-Mizrachi I, Kimelman M, Pruitt KD, Resenchuk S, Tatusova T, Yaschenko E, Ostell J. BioProject and BioSample databases at NCBI: facilitating capture and organization of metadata. Nucleic Acids Res. Jan;40(Database issue):D57-63 (2012). https://doi.org/10.1093/nar/gkr1163

work page doi:10.1093/nar/gkr1163 2012
[7]

Minimum information about a microarray experiment (MIAME)-toward standards for microarray data

Brazma A, Hingamp P, Quackenbush J, Sherlock G, Spellman P, Stoeckert C, Aach J, Ansorge W, Ball CA, Causton HC, Gaasterland T, Glenisson P, Holstege FC, Kim IF, Markowitz V , Matese JC, Parkinson H, Robinson A, Sarkans U, Schulze-Kremer S, Stewart J, Taylor R, Vilo J, Vingron M. Minimum information about a microarray experiment (MIAME)-toward standards f...

doi:10.1038/ng1201-365 2001
[8]

Genome biology 13(1), 5 (2012)

Mungall, C.J., Torniai, C., Gkoutos, G.V ., Lewis, S.E., Haendel, M.A.: Uberon, an integrative multi-species anatomy ontology. Genome biology 13(1), 5 (2012)

work page 2012

Show all 21 references

[9]

The center for expanded data annotation and retrieval

Musen MA, Bean CA, Cheung KH, Dumontier M, Durante KA, Gevaert O, Gonzalez-Beltran A, Khatri P, Kleinstein SH, O'Connor MJ, Pouliot Y , Rocca-Serra P, Sansone SA, Wiser JA; CEDAR team. The center for expanded data annotation and retrieval. J Am Med Inform Assoc. Nov;22(6):1148...

2015 doi
[10]

Jennifer Vendetti, Nomi L Harris, Michael V Dorf, Alex Skrenchuk, J Harry Caufield, Rafael S Gonçalves, John B Graybeal, Harshad Hegde, Timothy Redmond, Christopher J Mungall, Mark A Musen, BioPortal: an open community resource for sharing, searching, and utilizing biomedical ...

2025 doi
[11]

Gonçalves, R.S., Kamdar, M.R., Musen, M.A. (2019). Aligning Biomedical Metadata with Ontologies Using Clustering and Embeddings. In: Hitzler, P., et al. The Semantic Web. Lecture Notes in Computer Science(), vol 11503. Springer, Cham. ESWC (2019)

2019
[12]

Musen, M.A., O’Connor, M.J., Schultes, E. et al. Modeling community standards for metadata as templates makes data FAIR. Sci Data 9, 696 (2022). https://doi.org/10.1038/ s41597-022-01815-3

2022
[13]

Nature Communications 15(1), 1418 (2024) https://doi.org/10.1038/s41467-024-45563-x

Dagdelen, J., Dunn, A., Lee, S., Walker, N., Rosen, A.S., Ceder, G., Persson, K.A., Jain, A.: Structured information extraction from scientific text with large language models. Nature Communications 15(1), 1418 (2024) https://doi.org/10.1038/s41467-024-45563-x

2024 doi
[14]

bioRxiv(2025) https://doi.org/10.1101/2025.02.17.638570

Ikeda, S., Zou, Z., Bono, H., Moriya, Y ., Kawashima, S., Katayama, T., Oki, S., Ohta, T.: Extraction of biological terms using large language models enhances the usability of metadata in the biosample database. bioRxiv(2025) https://doi.org/10.1101/2025.02.17.638570

2025 doi
[15]

Sundaram, Benjamin Solomon, Avani Khatri, Anisha Laumas, Purvesh Khatri, Mark A

Sowmya S. Sundaram, Benjamin Solomon, Avani Khatri, Anisha Laumas, Purvesh Khatri, Mark A. Musen: Structured knowledge base enhances metadata curation by large language models. AMIA Annual Symposium (2024)

2024
[16]

Nucleic acids research 30(1), 207–210 (2002)

Edgar, R., Domrachev, M., Lash, A.E.: Gene expression omnibus: Ncbi gene expression and hybridization array data repository. Nucleic acids research 30(1), 207–210 (2002)

2002
[17]

al.: Llama 3: Open and efficient foundation language models

Touvron, H., et. al.: Llama 3: Open and efficient foundation language models. arXiv preprint arXiv:2308.12001 (2023)

2023 arXiv
[18]

al.: Medllama: A large language model for medicine

Alvi, O., et. al.: Medllama: A large language model for medicine. arXiv preprint arXiv:2308.09109 (2023)

2023 arXiv
[20]

Scientific Data 11(1), 268 (2024)

Dessimoz, C., Thomas, P.D.: Ai and the democratization of knowledge. Scientific Data 11(1), 268 (2024)

2024
[21]

Advances in Neural Information Processing Systems 36 (2024)

Lu, P., Peng, B., Cheng, H., Galley, M., Chang, K.-W., Wu, Y .N., Zhu, S.-C., Gao, J.: Chameleon: Plug-and-play compositional reasoning with large language models. Advances in Neural Information Processing Systems 36 (2024)

2024
[22]

Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing

Liu, Pengfei, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. "Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing." ACM computing surveys 55, no. 9 (2023)

2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Scientific data 3(1), 1–9 (2016)

Wilkinson, M.D., Dumontier, M., Aalbersberg, I.J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., Silva Santos, L.B., Bourne, P.E., et al.: The FAIR Guiding Principles for Scientific Data Management and Stewardship. Scientific data 3(1), 1–9 (2016)

work page 2016

[2] [2]

Nature 609(7926), 222–222 (2022)

Musen, M.A.: Without Appropriate Metadata, Data-Sharing Mandates are pointless. Nature 609(7926), 222–222 (2022)

work page 2022

[3] [3]

The variable quality of metadata about biological samples used in biomedical experiments

Gonçalves, R., Musen, M. The variable quality of metadata about biological samples used in biomedical experiments. Sci Data 6, 190021 (2019)

work page 2019

[4] [4]

& Musen, M.A

Miron, L., Gonçalves, R.S. & Musen, M.A. Obstacles to the reuse of study metadata in ClinicalTrials.gov. Sci Data 7, 443 (2020). https://doi.org/10.1038/s41597-020-00780-z

work page doi:10.1038/s41597-020-00780-z 2020

[5] [5]

arXiv preprint arXiv:2303.08774 (2023)

Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: GPT-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

arXiv 2023

[6] [6]

BioProject and BioSample databases at NCBI: facilitating capture and organization of metadata

Barrett T, Clark K, Gevorgyan R, Gorelenkov V , Gribov E, Karsch-Mizrachi I, Kimelman M, Pruitt KD, Resenchuk S, Tatusova T, Yaschenko E, Ostell J. BioProject and BioSample databases at NCBI: facilitating capture and organization of metadata. Nucleic Acids Res. Jan;40(Database issue):D57-63 (2012). https://doi.org/10.1093/nar/gkr1163

work page doi:10.1093/nar/gkr1163 2012

[7] [7]

Minimum information about a microarray experiment (MIAME)-toward standards for microarray data

Brazma A, Hingamp P, Quackenbush J, Sherlock G, Spellman P, Stoeckert C, Aach J, Ansorge W, Ball CA, Causton HC, Gaasterland T, Glenisson P, Holstege FC, Kim IF, Markowitz V , Matese JC, Parkinson H, Robinson A, Sarkans U, Schulze-Kremer S, Stewart J, Taylor R, Vilo J, Vingron M. Minimum information about a microarray experiment (MIAME)-toward standards f...

doi:10.1038/ng1201-365 2001

[8] [8]

Genome biology 13(1), 5 (2012)

Mungall, C.J., Torniai, C., Gkoutos, G.V ., Lewis, S.E., Haendel, M.A.: Uberon, an integrative multi-species anatomy ontology. Genome biology 13(1), 5 (2012)

work page 2012