Pith. sign in

REVIEW 4 major objections 5 minor 73 references

Assessing the Impact of the Quality of Textual Data on Feature Representation and Machine Learning Models

T0 review · 4 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Machine-learning models for clinical text tolerate token-level errors below 10 percent, but performance drops sharply once error rates reach 10 percent or higher.

desk verdict Useful empirical threshold for clinical text quality, but the exact 10% boundary is statistically under-supported and the 'error-free' baseline is assumed, not measured. read the letter →

arxiv 2502.08669 v1 pith:EQMPVVFR submitted 2025-02-12 cs.CL

Tabinda Sarwar , Antonio Jose Jimeno Yepes , Lawrence Cavedon This is my paper · ORCID
classification cs.CL
keywords clinicaltextdataqualitytoken-levelerrorrateMIMIC-IIIlargelanguagemodelcorrectionTF-IDFfeaturesBERTembeddingsmortalitypredictionagedcareprogressnotes
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether messy clinical free text can still support machine-learning predictions and where the tolerance limit lies. It defines error rate as the share of tokens with spelling or missing-whitespace errors, uses MIMIC-III as a clean baseline and an aged-care progress-note dataset as real-world noisy data, and uses the Mixtral LLM to inject, count, and correct errors. Across mortality, depression, and fall-risk tasks, models held their ROC-AUC when error rates stayed under 10 percent and lost ground once rates hit 10 percent or higher. The practical point is that a text dataset can be audited before modeling, and cleaning is only mandatory above the threshold.

What carries the argument

The load-bearing object is a token-level error rate metric: the number of erroneous tokens divided by the total number of tokens, ignoring grammar and punctuation. Alongside it, the paper uses Mixtral 8x7B as a multipurpose tool for quantifying errors, generating misspelled counterparts, and correcting text in context. Together, these turn data quality into a controllable variable: the authors inject known rates into clean MIMIC notes, measure the real-world aged-care baseline, then train classifiers on TF-IDF, word2vec, and BERT features with logistic regression, random forests, and XGBoost.

What would settle it

Manually audit a random sample of untouched MIMIC-III progress notes with the same token-level ground-truth protocol; if the baseline error rate is nonzero, recalculate the injected rates and check whether the performance curve still bends at 10 percent. Also run injections at 7, 8, 9, 10, and 11 percent to determine whether the decline is a sharp threshold or a gradual slope.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is a tolerance threshold: predictive models built on clinical free text are robust to small amounts of token-level noise, but cross a point near 10 percent injected errors where feature representations and classifiers lose accuracy. The authors reach this by injecting misspellings and missing whitespace into MIMIC-III at 5%, 10%, 15%, and 20%, while also correcting the aged-care dataset's measured 7.5% error rate with Mixtral; the absence of a major performance jump after correction is consistent with sub-10% noise. TF-IDF features outperform word2vec and BERT embeddings on both tasks, and the best classifier differs by dataset, with logistic regression leading on MIMIC and XGBoost leading on the aged-care data. Mixtral's error detection is imperfect, over-flagging medical terms and occasionally hallucinating errors, yet it matches the manually constructed ground truth within about three percentage points on a 136-note sample.

Load-bearing premise

The load-bearing premise is that MIMIC-III's progress notes are truly error-free and that Mixtral's error counts are accurate enough, so the injected rates are the real error rates.

Editorial extensions

If this is right

  • Datasets with measured token error rates under 10 percent can be fed into predictive models without corrective preprocessing, saving compute and time.
  • Datasets at or above 10 percent should be cleaned before modeling, because the reported AUC loss becomes substantial and task-dependent.
  • A token-level error audit can run before model development and serve as a cheap quality gate, using an LLM to count errors.
  • TF-IDF-based classifiers are a safer default for noisy clinical text than embedding features, since embeddings propagate and dilute token errors.
  • For real-world notes near 8 percent error, LLM correction changes the vocabulary and feature space but does not move predictions much, so cleaning very low-error text may have limited payoff.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the 10 percent threshold generalizes, data-quality budgets should target corpora above it; below that threshold, cleaning effort may be better spent elsewhere, but this is an editorial extension rather than a paper claim.
  • The threshold likely depends on error type: a misspelling that changes a word's meaning, such as a negation or a medication name, could hurt more than a missing space, so future work should weight errors by semantic severity rather than count tokens.
  • Because the paper's metric ignores grammar and punctuation, a dataset measured at under 10 percent token errors could still be low quality for syntax-sensitive tasks like relation extraction or question answering, and the threshold may not transfer to those settings.
  • A testable extension would be to inject semantically meaningful errors at 5 to 10 percent and compare AUC drops with random spelling errors, to see whether the tolerated band shrinks when mistakes change meaning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies how token-level textual errors affect feature representations and machine-learning classifiers for clinical text. It defines an error rate as the fraction of tokens with spelling or missing-whitespace errors, uses Mixtral to inject errors into the MIMIC-III mortality-prediction corpus at nominal rates of 5% to 20%, and also uses Mixtral to correct a private aged-care (ACH) corpus for depression and fall-risk prediction. The central empirical claim is that predictive performance (ROC-AUC) is tolerant to error rates below 10% but declines substantially at rates at or above 10%, and that the ACH corpus has an error rate of about 7.5-8%, so its performance is not severely degraded. The paper also reports that TF-IDF features outperform word2vec and BERT embeddings in these tasks.

Significance. If the 10% threshold is valid, the paper would provide practically useful guidance on when to invest in text cleaning for clinical NLP, and the combination of synthetic corruption and real-world LLM correction is a timely experimental design. The study is one of the few to systematically vary token-level noise in clinical text and to compare multiple feature representations and classifiers. Its strengths include a real-world low-quality dataset, a manual ground-truth evaluation of the LLM error detector, and transparent reporting of feature-count changes with error rate. However, the headline quantitative boundary is not currently supported by statistical evidence, the synthetic error-injection procedure appears to miscalibrate the actual error rates, and the two zero-error anchors of the analysis (original MIMIC and Mixtral-corrected ACH) are unverified. These issues limit the reliability of the central claim as stated.

major comments (4)
  1. [3.2.4, Eq. (1)] The synthetic error injection does not preserve the denominator of Eq. (1). When missing-whitespace errors are created by merging two adjacent tokens, the total number of tokens in the note decreases while the merged token is counted as a single erroneous token. If E tokens are selected for corruption and m of those are merges, the actual error rate becomes E / (N - m), which is larger than the nominal rate r = E/N. Since Figure 3 and the abstract report tolerance at nominal '<10%' and '>=10%', the x-axis is likely shifted right relative to the paper's own metric. The authors should compute and report the actual error rates of the corrupted notes after injection (using the same tokenization and counting procedure) and either adjust the injection algorithm or replot the performance curves against the measured rates.
  2. [4.2.1, Figure 3] The claim that performance 'declined significantly' for error rates >=10% is not supported by any inferential statistics. The figure shows ROC-AUC values at five discrete noise levels with no confidence intervals, error bars, or repeated runs. The differences between consecutive conditions appear small (often a few points of AUC), and no test establishes that the <10% versus >=10% comparison is significant. The authors should add variability estimates (e.g., bootstrap over patients or repeated corruption runs) and perform paired significance tests comparing the 0%/5% conditions with the 10%/15%/20% conditions. Without this, the specific 10% cutoff is an informal observation rather than an established result.
  3. [3.2.1 and 4.1.2] Both anchors of the error-rate scale are assumed rather than measured. The MIMIC corpus is declared 'error-free' because no errors have been reported, and the 'corrected' ACH dataset is produced by Mixtral, which the paper itself shows identifies all misspellings correctly in only 45% of notes and has a per-note deviation of 2.95% +/- 5.67% in estimated error rate. If MIMIC already contains a small percentage of typos or missing spaces, or if the corrected ACH still contains residual errors, the reported boundary of '<10% vs >=10%' is miscalibrated. The authors should quantify residual error rates in both corpora by sampling and manual inspection, and they should discuss how a nonzero baseline would shift the threshold claim.
  4. [4.1.2 vs Abstract/Discussion] The reporting of Mixtral's detection accuracy is internally inconsistent. Section 4.1.2 states that Mixtral identified the correct misspelt tokens for 45% of the progress notes, while the Abstract and Discussion report that Mixtral 'correctly detected errors in 63% of progress notes'. These are different metrics: the former likely means all misspellings in a note were correctly identified, whereas the latter might mean at least one error was detected. The authors should define both metrics explicitly and use them consistently throughout the paper, because the 63% vs 45% discrepancy directly affects how readers interpret the reliability of the error quantification and correction pipeline.
minor comments (5)
  1. [Throughout] The abbreviation 'TD-IDF' appears in several places (e.g., Section 3.4.1 and figure captions) and should be corrected to 'TF-IDF'.
  2. [Section 5] The sentence 'For the MIMIC dataset, linear regression shows higher prediction performance with TF-IDF features' should refer to 'logistic regression', since that is the classifier used in the experiments.
  3. [Section 3.2.4] The text says 'as this pattern was observed in real-life datasets (Table 2)', but the examples of missing whitespace are in Table 1. The cross-reference should be corrected.
  4. [Table S7] Table S7 lists the BERT feature dimensionality as 786, while the main text (Section 3.4.1) states that BERT embeddings have 768 dimensions. One of these is a typo and should be corrected.
  5. [Section 4.1.2] The sentence 'A performance deviation of 2.95 %±5.67% ... was observed' would be clearer if it specified whether this is the mean absolute deviation or the signed deviation of Mixtral's error rate from ground truth.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central error-tolerance result is an empirical measurement on synthetically corrupted text, anchored by a manual ground-truth check.

full rationale

The paper's central claim (models tolerate token-level error rates below 10% but degrade at or above 10%) is an empirical finding: errors are injected into MIMIC-III at controlled nominal rates of 5, 10, 15, and 20%, and downstream ROC-AUC is measured across TF-IDF, word2vec, and BERT representations with several classifiers. The independent variable (error rate, Eq. 1) is set by construction in the synthetic experiments, but the dependent variable (model performance) is measured independently, so the relationship is not a fitted parameter renamed as a prediction. Mixtral's error quantification is partially validated against a manually corrected 136-note ground truth (Section 4.1.2), which breaks the potential loop of using the same LLM to corrupt, quantify, and correct text. The discussion of limitations explicitly acknowledges that the error metric is rudimentary and that not all error types were quantified, which is a validity caveat rather than circular reasoning. The main weaknesses are calibration concerns, not circularity: MIMIC-III is assumed error-free without direct measurement (Section 3.2.1), and the ACH 'corrected' dataset is Mixtral-generated without independent verification of zero remaining errors. These issues affect the absolute position of the 10% threshold on a true error-rate axis, but they do not make any equation or result equivalent to its own input. Self-citations in the paper (e.g., reference [3] for EHR data-mining challenges) are contextual and not load-bearing for the central conclusion. Therefore, no circular step can be exhibited, and the appropriate score is 0.

Assumptions & free parameters 4 free parameters · 7 assumptions · 0 invented entities

The central claim rests on hand-chosen design decisions (feature thresholds, corruption oversampling, cohort filters) and on several domain assumptions about dataset cleanliness and label correctness. No new physical or conceptual entities are introduced. The absence of statistical error bars is the largest unstated cost in the ledger.

free parameters (4)
  • Minimum token frequency threshold for TF-IDF (min) = 5
    Chosen by hand in Section 3.4.1 to remove rare unique words; changes feature dimensionality and could affect how sensitive the error-rate curves appear.
  • Top-token count for TF-IDF (top) = 5000
    Fixed vocabulary size selected in Section 3.4.1; affects feature representation results.
  • Punctuation-token oversampling probability = 50% higher than non-punctuation tokens
    Section 3.2.4 states that tokens containing punctuation were 50% more likely to be selected for missing-space errors; this shapes the synthetic noise distribution.
  • Minimum progress-note count per resident = 21
    Section 4.1.1 excludes residents with fewer than 21 progress notes; this changes cohort sizes and potentially the difficulty of the prediction tasks.
assumptions (7)
  • domain assumption MIMIC-III progress notes are error-free, providing a clean baseline for synthetic error injection.
    Section 3.2.1 states MIMIC was deemed error-free because no textual mistakes have been reported by any study.
  • domain assumption Mixtral's error-detection accuracy measured on 136 manually corrected ACH notes transfers to the full ACH dataset.
    Section 4.1.2 reports the ground-truth check on a small sample from which notes shorter than 5 tokens were excluded; the full-dataset error rate of 7.5% is estimated through the same LLM.
  • domain assumption Token-level spelling and missing-whitespace errors are the relevant dimensions of data quality for this analysis.
    Section 3.2.2 defines the error rate using only these two error types and explicitly disregards punctuation and grammatical errors.
  • domain assumption Depression and fall labels from the ACH dataset are accurate as extracted from medical history and incident reports.
    Section 3.3 constructs binary labels without chart-review validation or adjudication, and the paper does not report label quality checks.
  • domain assumption The MIMIC mortality cohort and preprocessing from Mahbub et al. [63] transfer to this study.
    Section 3.3 refers to [63] for sampling details instead of describing them in this paper.
  • domain assumption Pre-trained word2vec and Clinical BERT embeddings, without fine-tuning, are adequate for assessing the impact of errors on feature representation.
    Section 3.4.1 uses pre-trained embeddings and explains that retraining on erroneous data could make the models treat errors as valid tokens; this choice affects the comparison.
  • domain assumption ROC-AUC from a single 2/3-1/3 split is sufficient to compare conditions.
    Section 3.4.2 reports no cross-validation, confidence intervals, or significance testing, yet the conclusions rely on differences between error-rate conditions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Assessing the Impact of the Quality of Textual Data on Feature Representation and Machine Learning Models." pith.science (2026). https://pith.science/paper/EQMPVVFR

@misc{pith2026250208669,
  author       = {Pith},
  title        = {Pith review of: Assessing the Impact of the Quality of Textual Data on Feature Representation and Machine Learning Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EQMPVVFR}},
  note         = {Machine review of arXiv:2502.08669}
}
read the original abstract

Background: Data collected in controlled settings typically results in high-quality datasets. However, in real-world applications, the quality of data collection is often compromised. It is well established that the quality of a dataset significantly impacts the performance of machine learning models. Methods: A rudimentary error rate metric was developed to evaluate textual dataset quality at the token level. Mixtral Large Language Model (LLM) was used to quantify and correct errors in low quality datasets. The study analyzed two healthcare datasets: the high-quality MIMIC-III public hospital dataset and a lower-quality private dataset from Australian aged care homes. Errors were systematically introduced into MIMIC at varying rates, while the ACH dataset quality was improved using the LLM. Results: For the sampled 35,774 and 6,336 patients from the MIMIC and ACH datasets respectively, we used Mixtral to introduce errors in MIMIC and correct errors in ACH. Mixtral correctly detected errors in 63% of progress notes, with 17% containing a single token misclassified due to medical terminology. LLMs demonstrated potential for improving progress note quality by addressing various errors. Under varying error rates, feature representation performance was tolerant to lower error rates (<10%) but declined significantly at higher rates. Conclusions: The study revealed that models performed relatively well on datasets with lower error rates (<10%), but their performance declined significantly as error rates increased (>=10%). Therefore, it is crucial to evaluate the quality of a dataset before utilizing it for machine learning tasks. For datasets with higher error rates, implementing corrective measures is essential to ensure the reliability and effectiveness of machine learning models.

Figures

Figures reproduced from arXiv: 2502.08669 by the authors.

Figure 1
Figure 1. The framework for evaluating the impact of the quality of textual datasets on machine learning models. This involves (a) systematically degrading the high-quality data and (b) improving the low-quality data for the NLP feature representation and predictive models. Dataset from Aged Care Homes Australian Aged Care Homes (ACHs) regularly record resident data to comply with the quality standards set by the Australian A… view at source ↗
Figure 2
Figure 2. Mixtral’s performance on 136 ground truth ACH progress notes for quantifying the textual errors (a) Number of tokens found in the ground truth progress notes. (b) Comparison of the error rate (%) found by Mixtral with the ground truth error rate. (c) The absolute error between the Mixtral computed and ground truth error rate. (d) The number of tokens that were erroneously detected or missed by Mixtral [PITH_FULL_IM… view at source ↗
Figure 3
Figure 3. clearly illustrates that, as hypothesized, the performance of the trained models declined as the level of errors in the dataset increased. A slight exception was observed with the XGBoost model for TF-IDF (raw) [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The performance of machine learning models for real-world ACH dataset using different feature representation techniques. The original and Mixtral’s corrected progress notes were used for predicting depression under (a) 30 days, (b) 60 days, and (c) 90 days tracking per…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

73 extracted references · 62 canonical work pages

  1. [1]

    A guide to deep learning in healthcare

    Esteva A, Robicquet A, Ramsundar B, et al. A guide to deep learning in healthcare. Nat Med 2019; 25: 24–29

    work page 2019

  2. [2]

    Mining electronic health records: towards better research applications and clinical care

    Jensen PB, Jensen LJ, Brunak S. Mining electronic health records: towards better research applications and clinical care. Nat Rev Genet 2012; 13: 395– 405

    work page 2012

  3. [3]

    The Secondary Use of Electronic Health Records for Data Mining: Data Characteristics and Challenges

    Sarwar T, Seifollahi S, Chan J, et al. The Secondary Use of Electronic Health Records for Data Mining: Data Characteristics and Challenges. ACM Comput Surv 2022; 55: 33:1-33:40

    work page 2022

  4. [4]

    Mining Electronic Health Records (EHRs): A Survey

    Yadav P, Steinbach M, Kumar V, et al. Mining Electronic Health Records (EHRs): A Survey. ACM Comput Surv 2018; 50: 85:1-85:40

    work page 2018

  5. [5]

    Impact of Different Approaches to Preparing Notes for Analysis With Natural Language Processing on the Performance of Prediction Models in Intensive Care

    Mahendra M, Luo Y, Mills H, et al. Impact of Different Approaches to Preparing Notes for Analysis With Natural Language Processing on the Performance of Prediction Models in Intensive Care. Critical Care Explorations 2021; 3: e0450

    work page 2021

  6. [6]

    An advanced review on text mining in medicine

    Luque C, Luna JM, Luque M, et al. An advanced review on text mining in medicine. WIREs Data Mining and Knowledge Discovery 2019; 9: e1302

    work page 2019

  7. [7]

    Natural Language Processing for EHR-Based Computational Phenotyping

    Zeng Z, Deng Y, Li X, et al. Natural Language Processing for EHR-Based Computational Phenotyping. IEEE/ACM Transactions on Computational Biology and Bioinformatics 2019; 16: 139–153

    work page 2019

  8. [8]

    Clinical Text Data in Machine Learning: Systematic Review

    Spasic I, Nenadic G. Clinical Text Data in Machine Learning: Systematic Review. JMIR Medical Informatics 2020; 8: e17984

    work page 2020

Show all 73 references
  1. [9]

    Attention is all you need

    Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY, USA: Curran Associates Inc., 2017, pp. 6000–6010

    2017

  2. [10]

    Transformers for Multi-label Classification of Medical Text: An Empirical Comparison

    Yogarajan V, Montiel J, Smith T, et al. Transformers for Multi-label Classification of Medical Text: An Empirical Comparison. In: Tucker A, Henriques Abreu P, Cardoso J, et al. (eds) Artificial Intelligence in Medicine. Cham: Springer International Publishing, 2021, pp. 114–123

    2021

  3. [11]

    Measurement of Semantic Textual Similarity in Clinical Texts: Comparison of Transformer-Based Models

    Yang X, He X, Zhang H, et al. Measurement of Semantic Textual Similarity in Clinical Texts: Comparison of Transformer-Based Models. JMIR Medical Informatics 2020; 8: e19735

    2020

  4. [12]

    Limitations of Transformers on Clinical Text Classification

    Gao S, Alawad M, Young MT, et al. Limitations of Transformers on Clinical Text Classification. IEEE Journal of Biomedical and Health Informatics 2021; 25: 3596–3607

    2021

  5. [13]

    A Multimodal Transformer: Fusing Clinical Notes with Structured EHR Data for Interpretable In-Hospital Mortality Prediction

    Lyu W, Dong X, Wong R, et al. A Multimodal Transformer: Fusing Clinical Notes with Structured EHR Data for Interpretable In-Hospital Mortality Prediction. AMIA Annu Symp Proc 2023; 2022: 719–728

    2023

  6. [14]

    A Survey of Large Language Models

    Zhao WX, Zhou K, Li J, et al. A Survey of Large Language Models. Epub ahead of print 13 October 2024. DOI: 10.48550/arXiv.2303.18223

    2024 doi arXiv

  7. [15]

    A Survey on Evaluation of Large Language Models

    Chang Y, Wang X, Wang J, et al. A Survey on Evaluation of Large Language Models. ACM Trans Intell Syst Technol 2024; 15: 39:1-39:45

    2024

  8. [16]

    Hadi MU, Tashi QA, Qureshi R, et al. A Survey on Large Language Models: Applications, Challenges, Limitations, and Practical Usage, https://www.authorea.com/doi/full/10.36227/techrxiv.23589741.v1?commi t=b1cb46f5b0f749cf5f2f33806f7c124904c14967 (accessed 4 December 2024)

    doi

  9. [17]

    Large Language Models: A Survey

    Minaee S, Mikolov T, Nikzad N, et al. Large Language Models: A Survey. Epub ahead of print 20 February 2024. DOI: 10.48550/arXiv.2402.06196

    2024 doi arXiv

  10. [18]

    Attention is not all you need: the complicated case of ethically using large language models in healthcare and medicine

    Harrer S. Attention is not all you need: the complicated case of ethically using large language models in healthcare and medicine. eBioMedicine; 90. Epub ahead of print 1 April 2023. DOI: 10.1016/j.ebiom.2023.104512

    2023

  11. [19]

    BioBERT: a pre-trained biomedical language representation model for biomedical text mining

    Lee J, Yoon W, Kim S, et al. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 2020; 36: 1234–1240

    2020

  12. [20]

    ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission

    Huang K, Altosaar J, Ranganath R. ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission. Epub ahead of print 29 November 2020. DOI: 10.48550/arXiv.1904.05342

    2020 doi arXiv

  13. [21]

    Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing

    Gu Y, Tinn R, Cheng H, et al. Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing. ACM Trans Comput Healthcare 2021; 3: 2:1-2:23

    2021

  14. [22]

    Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks

    Gururangan S, Marasović A, Swayamdipta S, et al. Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks. In: Jurafsky D, Chai J, Schluter N, et al. (eds) Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Online: Association fo...

  15. [23]

    SciBERT: A Pretrained Language Model for Scientific Text

    Beltagy I, Lo K, Cohan A. SciBERT: A Pretrained Language Model for Scientific Text. In: Inui K, Jiang J, Ng V, et al. (eds) Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Pro...

    2019

  16. [24]

    MedCPT: Contrastive Pre-trained Transformers with large-scale PubMed search logs for zero-shot biomedical information retrieval

    Jin Q, Kim W, Chen Q, et al. MedCPT: Contrastive Pre-trained Transformers with large-scale PubMed search logs for zero-shot biomedical information retrieval. Bioinformatics 2023; 39: btad651

    2023

  17. [25]

    A large language model for electronic health records

    Yang X, Chen A, PourNejatian N, et al. A large language model for electronic health records. npj Digit Med 2022; 5: 1–9

    2022

  18. [26]

    MIMIC-III, a freely accessible critical care database

    Johnson AEW, Pollard TJ, Shen L, et al. MIMIC-III, a freely accessible critical care database. Sci Data 2016; 3: 160035

    2016

  19. [27]

    Annotating longitudinal clinical narratives for de- identification: The 2014 i2b2/UTHealth corpus

    Stubbs A, Uzuner Ö. Annotating longitudinal clinical narratives for de- identification: The 2014 i2b2/UTHealth corpus. Journal of Biomedical Informatics 2015; 58: S20–S29

    2014

  20. [28]

    Automated systems for the de-identification of longitudinal clinical narratives: Overview of 2014 i2b2/UTHealth shared task Track 1

    Stubbs A, Kotfila C, Uzuner Ö. Automated systems for the de-identification of longitudinal clinical narratives: Overview of 2014 i2b2/UTHealth shared task Track 1. Journal of Biomedical Informatics 2015; 58: S11–S19

    2014

  21. [29]

    Overview and Importance of Data Quality for Machine Learning Tasks

    Jain A, Patel H, Nagalapatti L, et al. Overview and Importance of Data Quality for Machine Learning Tasks. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York, NY, USA: Association for Computing Machinery, pp. 3561–3562

  22. [30]

    Data Quality Toolkit: Automatic assessment of data quality and remediation for machine learning datasets

    Gupta N, Patel H, Afzal S, et al. Data Quality Toolkit: Automatic assessment of data quality and remediation for machine learning datasets. Epub ahead of print 5 September 2021. DOI: 10.48550/arXiv.2108.05935

    2021 doi arXiv

  23. [31]

    Data Validation for Machine Learning

    Polyzotis N, Zinkevich M, Roy S, et al. Data Validation for Machine Learning. Proceedings of Machine Learning and Systems 2019; 1: 334–347

    2019

  24. [32]

    Data Evaluation and Enhancement for Quality Improvement of Machine Learning

    Chen H, Chen J, Ding J. Data Evaluation and Enhancement for Quality Improvement of Machine Learning. IEEE Transactions on Reliability 2021; 70: 831–847

    2021

  25. [33]

    Data quality considerations for big data and machine learning: Going beyond data cleaning and transformations

    Gudivada V, Apon A, Ding J. Data quality considerations for big data and machine learning: Going beyond data cleaning and transformations. International Journal on Advances in Software 2017; 10: 1–20

    2017

  26. [34]

    Data Readiness Report

    Afzal S, Rajmohan C, Kesarwani M, et al. Data Readiness Report. In: 2021 IEEE International Conference on Smart Data Services (SMDS), pp. 42–51

    2021

  27. [35]

    Executing Data Quality Projects Ten Steps to Quality Data and Trusted Information (TM)

    McGilvray D. Executing Data Quality Projects Ten Steps to Quality Data and Trusted Information (TM). Second. 2021

    2021

  28. [36]

    A Short Review of the Literature on Automatic Data Quality

    Chandran DR, Gupta V. A Short Review of the Literature on Automatic Data Quality. Journal of Computer and Communications 2022; 10: 55–73

    2022

  29. [37]

    A Practical Framework for Evaluating the Quality of Knowledge Graph

    Chen H, Cao G, Chen J, et al. A Practical Framework for Evaluating the Quality of Knowledge Graph. In: Zhu X, Qin B, Zhu X, et al. (eds) Knowledge Graph and Semantic Computing: Knowledge Computing and Language Understanding. Singapore: Springer, 2019, pp. 111–122

    2019

  30. [38]

    Developing a systematic approach to assessing data quality in secondary use of clinical data based on intended use

    Razzaghi H, Greenberg J, Bailey LC. Developing a systematic approach to assessing data quality in secondary use of clinical data based on intended use. Learning Health Systems 2022; 6: e10264

    2022

  31. [39]

    Defining and measuring completeness of electronic health records for secondary use

    Weiskopf NG, Hripcsak G, Swaminathan S, et al. Defining and measuring completeness of electronic health records for secondary use. Journal of Biomedical Informatics 2013; 46: 830–836

    2013

  32. [40]

    Review: Electronic Health Records and the Reliability and Validity of Quality Measures: A Review of the Literature

    Chan KS, Fowles JB, Weiner JP. Review: Electronic Health Records and the Reliability and Validity of Quality Measures: A Review of the Literature. Med Care Res Rev 2010; 67: 503–527

    2010

  33. [41]

    Quality Indicators for Text Data

    Kiefer C. Quality Indicators for Text Data. Gesellschaft für Informatik, Bonn, pp. 145–154

  34. [42]

    Outlier Detection for Improved Data Quality and Diversity in Dialog Systems

    Larson S, Mahendran A, Lee A, et al. Outlier Detection for Improved Data Quality and Diversity in Dialog Systems. In: Burstein J, Doran C, Solorio T (eds) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human L...

    2019

  35. [43]

    Aiming beyond the Obvious: Identifying Non- Obvious Cases in Semantic Similarity Datasets

    Peinelt N, Liakata M, Nguyen D. Aiming beyond the Obvious: Identifying Non- Obvious Cases in Semantic Similarity Datasets. In: Korhonen A, Traum D, Màrquez L (eds) Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: Associa...

  36. [44]

    Evolutionary Data Measures: Understanding the Difficulty of Text Classification Tasks

    Collins E, Rozanov N, Zhang B. Evolutionary Data Measures: Understanding the Difficulty of Text Classification Tasks. In: Korhonen A, Titov I (eds) Proceedings of the 22nd Conference on Computational Natural Language Learning. Brussels, Belgium: Association for Computational L...

  37. [45]

    Improving Neural Response Diversity with Frequency-Aware Cross-Entropy Loss

    Jiang S, Ren P, Monz C, et al. Improving Neural Response Diversity with Frequency-Aware Cross-Entropy Loss. In: The World Wide Web Conference. New York, NY, USA: Association for Computing Machinery, pp. 2879–2885

  38. [46]

    Confident Learning: Estimating Uncertainty in Dataset Labels

    Northcutt C, Jiang L, Chuang I. Confident Learning: Estimating Uncertainty in Dataset Labels. J Artif Int Res 2021; 70: 1373–1411

    2021

  39. [47]

    Detecting errors in part-of-speech annotation

    Dickinson M, Meurers WD. Detecting errors in part-of-speech annotation. In: Proceedings of the tenth conference on European chapter of the Association for Computational Linguistics - Volume 1. USA: Association for Computational Linguistics, pp. 107–114

  40. [48]

    Detecting errors within a corpus using anomaly detection

    Eskin E. Detecting errors within a corpus using anomaly detection. In: Proceedings of the 1st North American chapter of the Association for Computational Linguistics conference. USA: Association for Computational Linguistics, 2000, pp. 148–153

    2000

  41. [49]

    Detecting annotation noise in automatically labelled data

    Rehbein I, Ruppenhofer J. Detecting annotation noise in automatically labelled data. In: Barzilay R, Kan M-Y (eds) Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Vancouver, Canada: Association for Computational ...

  42. [50]

    Data Quality for Machine Learning Tasks

    Gupta N, Mujumdar S, Patel H, et al. Data Quality for Machine Learning Tasks. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. New York, NY, USA: Association for Computing Machinery, pp. 4040–4041

  43. [51]

    Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics

    Swayamdipta S, Schwartz R, Lourie N, et al. Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics. In: Webber B, Cohn T, He Y, et al. (eds) Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Online: Association...

    2020

  44. [52]

    Beyond Accuracy: Behavioral Testing of NLP Models with CheckList

    Ribeiro MT, Wu T, Guestrin C, et al. Beyond Accuracy: Behavioral Testing of NLP Models with CheckList. In: Jurafsky D, Chai J, Schluter N, et al. (eds) Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Online: Association for Computationa...

  45. [53]

    Impact of data quality for automatic issue classification using pre-trained language models

    Colavito G, Lanubile F, Novielli N, et al. Impact of data quality for automatic issue classification using pre-trained language models. Journal of Systems and Software 2024; 210: 111838

    2024

  46. [54]

    AutoCorrect, https://pypi.org/project/autocorrect/

  47. [55]

    Pyaspeller, https://pypi.org/project/pyaspeller/

  48. [56]

    Grammar Check, https://pypi.org/project/grammar-check/1.3.1/

  49. [57]

    A Comprehensive Survey of Grammatical Error Correction

    Wang Y, Wang Y, Dang K, et al. A Comprehensive Survey of Grammatical Error Correction. ACM Trans Intell Syst Technol 2021; 12: 65:1-65:51

    2021

  50. [58]

    The CoNLL-2014 Shared Task on Grammatical Error Correction

    Ng HT, Wu SM, Briscoe T, et al. The CoNLL-2014 Shared Task on Grammatical Error Correction. In: Ng HT, Wu SM, Briscoe T, et al. (eds) Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task. Baltimore, Maryland: Association for Computat...

    2014

  51. [59]

    Grammatical Error Correction: A Survey of the State of the Art

    Bryant C, Yuan Z, Qorib MR, et al. Grammatical Error Correction: A Survey of the State of the Art. Computational Linguistics 2023; 49: 643–701

    2023

  52. [60]

    The Battle of LLMs: A Comparative Study in Conversational QA Tasks

    Rangapur A, Rangapur A. The Battle of LLMs: A Comparative Study in Conversational QA Tasks. Epub ahead of print 28 May 2024. DOI: 10.48550/arXiv.2405.18344

    2024 doi arXiv

  53. [61]

    Beyond ChatGPT: Enhancing Software Quality Assurance Tasks with Diverse LLMs and Validation Techniques

    Widyasari R, Lo D, Liao L. Beyond ChatGPT: Enhancing Software Quality Assurance Tasks with Diverse LLMs and Validation Techniques. Epub ahead of print 2 September 2024. DOI: 10.48550/arXiv.2409.01001

    2024 doi arXiv

  54. [62]

    Mixtral of Experts

    Jiang AQ, Sablayrolles A, Roux A, et al. Mixtral of Experts. Epub ahead of print 8 January 2024. DOI: 10.48550/arXiv.2401.04088

    2024 doi arXiv

  55. [63]

    Unstructured clinical notes within the 24 hours since admission predict short, mid & long-term mortality in adult ICU patients

    Mahbub M, Srinivasan S, Danciu I, et al. Unstructured clinical notes within the 24 hours since admission predict short, mid & long-term mortality in adult ICU patients. PLOS ONE 2022; 17: e0262182

    2022

  56. [64]

    Mental health of older adults, https://www.who.int/news-room/fact- sheets/detail/mental-health-of-older-adults

  57. [65]

    WHO global report on falls prevention in older age, https://www.who.int/publications/i/item/9789241563536

  58. [66]

    Distributed representations of words and phrases and their compositionality

    Mikolov T, Sutskever I, Chen K, et al. Distributed representations of words and phrases and their compositionality. In: Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2. Red Hook, NY, USA: Curran Associates Inc., 2013, pp. 3111–3119

    2013

  59. [67]

    BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

    Devlin J, Chang M-W, Lee K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In: Burstein J, Doran C, Solorio T (eds) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Huma...

    2019

  60. [68]

    Publicly Available Clinical BERT Embeddings

    Alsentzer E, Murphy J, Boag W, et al. Publicly Available Clinical BERT Embeddings. In: Rumshisky A, Roberts K, Bethard S, et al. (eds) Proceedings of the 2nd Clinical Natural Language Processing Workshop. Minneapolis, Minnesota, USA: Association for Computational Linguistics, ...

  61. [69]

    Longformer: The Long-Document Transformer

    Beltagy I, Peters ME, Cohan A. Longformer: The Long-Document Transformer. Epub ahead of print 2 December 2020. DOI: 10.48550/arXiv.2004.05150. Supplementary Data Table S1: Sample queries demonstrating the extraction of tokens with errors using Mixtral. The blue text represents...

    2020 doi arXiv

  62. [70]

    Resident buzzed at2300hrs on5/10/10,her legs felt like they were burning n she was in pain

    Strictly proofread and correct any spelling mistakes in the following sentences. Only produce a single corrected text and do not provide any explanation. Return 'nil' if there is no mistake: “Resident buzzed at2300hrs on5/10/10,her legs felt like they were burning n she was in...

  63. [71]

    Resident was sleeping on round check,repositioned by2 x staff fluids given. nil problems settled ator

    Strictly proofread and correct any spelling mistakes in the following sentences. Only produce a single corrected text and do not provide any explanation. Return 'nil' if there is no mistake: “Resident was sleeping on round check,repositioned by2 x staff fluids given. nil probl...

  64. [72]

    Resident

    Correct any spelling mistakes in the following list. Only generate a list containing all the provided tokens along with the ones that have been corrected do not provide any explanation: ["Resident", "buzzed", "at2300hrs", "on5/10/10,her", "legs", "felt", "like", "they", "were"...

  65. [73]

    Resident

    Correct any spelling mistakes in the following list. Only generate a list containing all the provided tokens along with the ones that have been corrected do not provide any explanation: ["Resident", "was", "sleeping", "on", "round", "check,repositioned", "by2", "x", "staff", "...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.