Pith. sign in

REVIEW 4 major objections 5 minor 59 references

Semi-Supervised Learning from Small Annotated Data and Large Unlabeled Data for Fine-grained PICO Entity Recognition

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read An iterative self-training pipeline with pseudo-label quality filters reaches macro F1 0.60 for fine-grained PICO entity recognition using only 10% of the annotated training data, outperforming the supervised baseline by more than 16%.

desk verdict Useful SSL-for-PICO study whose headline gain is likely real in direction but inflated by test-set iteration selection and single-mask evaluation. read the letter →

arxiv 2412.19346 v1 pith:FDGKR7PI submitted 2024-12-26 cs.CL cs.AI

Fangyi Chen , Gongbo Zhang , Yilu Fang , Yifan Peng , Chunhua Weng This is my paper · ORCID
classification cs.CLcs.AI
keywords fine-grainedPICOextractionsemi-supervisedlearningnamedentityrecognitionself-trainingpseudo-labelingclinicaltrialabstractsBiomedBERTGPT-4olabelselection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper aims to show that a semi-supervised self-training loop can reduce the annotation burden for fine-grained PICO extraction from clinical trial abstracts without sacrificing accuracy. Using 2,511 abstracts from four public corpora, the authors train a BiomedBERT NER model on a small labeled subset, generate pseudo-labels for a larger unlabeled pool, and selectively retrain on the most confident pseudo-labels. With only 10% of the training sentences labeled, FinePICO reaches macro F1 0.60, compared with 0.437 for the fully supervised baseline, and the gains hold across a revised annotation scheme and external test corpora. The result matters because meta-analysts need fine-grained entities such as arm-specific sample sizes and outcome values, which are expensive to annotate and inconsistently defined across existing datasets.

What carries the argument

The load-bearing mechanism is an iterative self-training loop guarded by a pseudo-label quality function. A BiomedBERT model trained on labeled sentences predicts labels for unlabeled sentences; the quality function f in Eq. (8) decides which pseudo-labeled tokens enter the next training pool, and the model is retrained on the combined supervised and filtered unsupervised loss until validation performance converges. Three filters are tested: confident-based masking, which keeps tokens above a fixed softmax threshold; class-adaptive threshold masking, which sets a per-class threshold tau_k from the mean maximum probability of tokens predicted in that class; and GPT-4o-based selection, which prompts a large language model with entity-specific annotation guidelines to confirm each pseudo-label. The filter is what prevents the self-training loop from amplifying its own errors, and it is the component the paper varies to show the gain over the baseline is real.

What would settle it

Take the same 2,511-abstract corpus and repeat the 10% labeled-data experiment over 10 or more random masks; if the bootstrapped 95% confidence interval for FinePICO's macro F1 overlaps the baseline's 0.437, the annotation-saving claim fails. Also rerun the loop with the quality filter replaced by random acceptance of pseudo-labels at the same volume; if F1 does not fall below 0.60, the filter is not the source of the gain.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that iterative self-training with pseudo-label quality control lets fine-grained PICO NER approach the performance of fully supervised training with a fraction of the annotations. In the original PICO scheme with 10% of training sentences labeled, the best FinePICO configurations reach macro F1 0.606 (in-domain, class-adaptive threshold) and 0.60 (cross-domain, GPT-based selection), against a BiomedBERT baseline of 0.437; the improvement is statistically significant at p<0.001. The method also transfers to a revised PICO scheme, where the class-adaptive threshold reaches F1 0.653 with in-domain unlabeled data, and it improves partial-match F1 on the AD and COVID-19 corpora over the baseline. The authors interpret this as evidence that abundant unlabeled RCT abstracts can substitute for much of the missing manual annotation in fine-grained PICO extraction.

Load-bearing premise

The method assumes that tokens whose pseudo-labels pass the confidence filter (or GPT-4o confirmation) are correct often enough that adding them to the training pool improves the model, and it assumes the single random 10% label split behind the headline number is representative; if either gives way, the reported margin over the baseline shrinks or disappears.

Editorial extensions

If this is right

  • Annotation effort can drop by roughly 90%: with 10% labeled data, FinePICO's F1 of 0.60 exceeds the fully supervised baseline's 0.437, making fine-grained PICO extraction feasible for teams without large annotation budgets.
  • Cheap confidence filters can substitute for LLM-based label checking: the class-adaptive threshold matches or beats GPT-4o selection in the revised scheme and shows no statistically significant loss in the original scheme.
  • Cross-domain unlabeled data is at least as useful as in-domain data in the original scheme, so publicly available RCT abstracts from other specialties can serve as the unlabeled pool when in-domain abstracts are scarce.
  • The performance curve saturates as labeled data approaches 100%, indicating that the SSL gain is concentrated in low-resource regimes and that adding more unlabeled data cannot fully replace the last increments of annotation.
  • Because the method works under a revised PICO scheme, downstream users can define their own fine-grained entity taxonomy and still benefit from the same semi-supervised pipeline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The near-tie among the three filters suggests the pseudo-label loop, not the filter's sophistication, is the main driver; a testable extension is fixing the filter and varying retraining frequency to see whether one or two iterations capture most of the gain.
  • The most frequent error is boundary detection, and the model also confuses intervention-arm and control-arm values; this points to a document-level or section-aware extension that uses context beyond a single sentence.
  • If the result transfers to other clinical NER tasks, the same recipe—small labeled slice plus large unlabeled pool plus class-adaptive filtering—could lower annotation costs for eligibility criteria, adverse events, or drug-regimen extraction, where annotated corpora are similarly sparse.
  • GPT-4o confirmation is expensive per token; since confidence-based filters match it in F1, a practical next test is comparing cost-adjusted throughput of the three filters on a much larger unlabeled pool.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes FinePICO, an iterative self-training semi-supervised NER pipeline for fine-grained PICO entity extraction from clinical trial abstracts. The method trains a BiomedBERT baseline on a small annotated subset, generates pseudo-labels on a larger unlabeled set, and filters pseudo-labels using one of three quality-enhancement strategies: confident-based masking, class-adaptive thresholding, or GPT-4o-based confirmation. The main empirical claim is that with only 10% annotated training sentences, FinePICO reaches macro F1 0.600 on the PICO-Corpus test set, outperforming a supervised baseline at F1 0.437 by more than 16%. The paper also reports generalization to a revised PICO annotation scheme and to external AD/COVID-19 corpora, with p-values below 0.001 for several comparisons.

Significance. If the central quantitative claim holds, the paper offers practical evidence that semi-supervised self-training can reduce annotation cost for fine-grained PICO extraction while approaching fully supervised performance. The paper has several strengths: it uses publicly available corpora, evaluates against held-out test sets, reports bootstrapped confidence intervals, compares three pseudo-label selection mechanisms, and includes external-corpus generalization tests. However, the headline result is vulnerable to two evaluation-design issues: the best iteration appears to be selected from training curves without a clear validation-based stopping rule, and all 10%-annotation results are based on a single randomly masked subset with no variance across label subsamples. These issues must be resolved before the reported effect size can be considered reliable.

major comments (4)
  1. [§2.1, §3.1, Figure 3] The paper reports the 'highest F1' observed over iterations for each method, but it does not specify whether the iteration achieving that F1 was chosen on the validation set or by inspecting test-set performance. Section 2.1 states that training stops when performance converges on validation, which is a different rule from selecting the maximum over iterations; in self-training loops early iterations are often better than later ones, so taking the maximum over the displayed curves can materially inflate the reported gain. Please report the F1 obtained under a pre-specified validation-based stopping rule, or otherwise demonstrate that the reported numbers were not selected by peeking at the test set.
  2. [§2.4, Supplementary Table 3] All 10%-annotation results, including the headline F1 of 0.600 versus baseline 0.437, appear to come from a single randomly selected 10% labeled subset. No variance over different label masks is reported, so it is unclear whether the observed improvement reflects a favorable label draw or a systematic effect of the method. Please rerun the key comparison over at least five different random masks and report the mean and standard deviation of the performance gap, together with a paired significance test across masks.
  3. [§3.2, Supplementary Table 5] Several reported confidence intervals are internally inconsistent. For example, the class-adaptive in-domain row reports F1 = 0.596 with 95% CI (0.609, 0.664), and the GPT cross-domain row reports F1 = 0.600 with 95% CI (0.609, 0.664); in both cases the interval lies entirely above the point estimate and even excludes it. This suggests a column misalignment or a bootstrapping error. Because the statistical claims in Section 3.2 depend on these intervals, they need to be corrected and recomputed.
  4. [§2.1.3, Eq. (9)] The class-adaptive threshold in Eq. (9) is not fully specified: P(k|t_ij) is not defined, and the formula appears to divide a sum of maximum token probabilities by the total count of pseudo-labels per class, rather than computing a per-token confidence threshold. As written, it is unclear how a token is accepted or rejected under this rule, and the method is therefore not reproducible from the description. Please state the precise filtering condition, including how the threshold is applied to individual tokens.
minor comments (5)
  1. [Abstract, §4] The phrase 'by more than 16%' should be clarified: (0.600 - 0.437) is 16.3 percentage points, but the relative improvement is approximately 37%. Please state explicitly whether the comparison is in percentage points or relative percent to avoid misleading readers.
  2. [Figure 3 caption] The caption contains a typo: 'Lower bound performance is detonated as the baseline model' should read 'denoted.'
  3. [Table 1, Supplementary Table 3] The counts in Table 1 appear to mix abstract-level and sentence-level units: PICO-Corpus is listed as 1,011 abstracts, but Training/Validation/Test are given as 1,010/645/944, and Supplementary Table 3 later states the test corpus has 2,717 sentences. Please clarify the units and ensure the reported splits are consistent across tables.
  4. [§2.1, §4] Two important hyperparameters are never given concrete values: the confidence threshold used in confident-based masking and the unsupervised loss weight alpha in Eq. (4). Reporting the chosen values would improve reproducibility.
  5. [Data availability] The statement that 'data and codes underlying the study will be available upon request' is insufficient for a reproducible empirical paper; a permanent repository link would be expected.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: FinePICO's gains are measured against held-out test labels and external baselines, not derived from its own inputs.

full rationale

The paper's central claim is an empirical comparison: FinePICO trained on 10% labeled data plus unlabeled data reaches macro F1 0.60 versus the 0.437 supervised baseline (Abstract; Section 3.1; Table 2), with evaluation performed on held-out test sets using seqeval (Section 2.6). That evaluation is external to the training loop and does not reduce to any fitted quantity or to the pseudo-labels themselves. The self-training procedure in Eqs. 1-8 reuses the model's own predictions as pseudo-labels, but this is the algorithmic design being tested, not a circular derivation of the reported result; the quality-enhancement functions f and the class-adaptive threshold in Eq. 9 filter pseudo-labels, and any risk from incorrect pseudo-labels is an empirical limitation acknowledged in the Discussion, not a definitional equivalence. The 'lower' and 'upper' performance bounds are empirically trained supervised baselines (Sections 2.3 and 3.1), not theoretical bounds derived from the method's own outputs, so their use does not constitute circularity. The author-overlapping citations (e.g., [44], and background citations [4,5]) support foundation-model selection and prior context, but the load-bearing SSL improvement claim is justified by the held-out test comparisons and external AD/COVID-19 corpora, not by those citations; no uniqueness theorem or ansatz is imported from prior work by the same authors. The concern that the best iteration or best quality variant may have been selected using test-set performance, and the use of a single 10% label mask, is a statistical validity and reproducibility issue, not a circularity issue under the stated rules. No equation or fitted parameter is renamed as a prediction, and no step in the paper reduces by construction to its own inputs. Therefore the appropriate finding is no significant circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

This is an empirical ML paper with no mathematical derivation. The central claim rests on standard deep-learning assumptions, an implicit assumption that pseudo-label confidence tracks correctness, and unstated hyperparameters such as alpha and the confidence threshold.

free parameters (3)
  • Confidence threshold for confident-based masking = not reported
    The threshold filters pseudo-labels; the authors say it is empirically determined (Section 2.1.3), but no value or tuning procedure is provided.
  • Alpha (unsupervised loss weight) = not reported
    Equation (4) defines L_total = L_s + alpha L_u, but alpha is never specified. It controls how much influence pseudo-labeled tokens have on training.
  • Maximum iteration count = 10
    The loop runs up to 10 iterations, and the reported best F1 often occurs at iteration 7 or later, so the stopping rule materially affects the headline number.
assumptions (4)
  • domain assumption BiomedBERT provides token representations sufficient for PICO entity classification
    Used as the foundation model; pretraining on biomedical text is assumed to transfer to the PICO NER task (Section 2.3).
  • domain assumption High softmax confidence correlates with correct pseudo-labels
    The quality enhancement function f in Eq. 8 assumes that probability thresholding (or GPT-4o confirmation) selects accurate labels. If false, self-training amplifies errors.
  • domain assumption GPT-4o's yes/no verification of pseudo-labels matches human annotation
    GPT-based selection uses GPT-4o as an external judge (Section 2.1.3); no agreement study between GPT-4o and human labels is reported.
  • domain assumption EBM-NLP unlabeled abstracts are drawn from a distribution useful for the target PICO-Corpus task
    Cross-domain augmentation assumes relatedness; distribution shift is acknowledged as a known SSL failure mode in the Discussion.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Semi-Supervised Learning from Small Annotated Data and Large Unlabeled Data for Fine-grained PICO Entity Recognition." pith.science (2026). https://pith.science/paper/FDGKR7PI

@misc{pith2026241219346,
  author       = {Pith},
  title        = {Pith review of: Semi-Supervised Learning from Small Annotated Data and Large Unlabeled Data for Fine-grained PICO Entity Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FDGKR7PI}},
  note         = {Machine review of arXiv:2412.19346}
}
read the original abstract

Objective: Extracting PICO elements -- Participants, Intervention, Comparison, and Outcomes -- from clinical trial literature is essential for clinical evidence retrieval, appraisal, and synthesis. Existing approaches do not distinguish the attributes of PICO entities. This study aims to develop a named entity recognition (NER) model to extract PICO entities with fine granularities. Materials and Methods: Using a corpus of 2,511 abstracts with PICO mentions from 4 public datasets, we developed a semi-supervised method to facilitate the training of a NER model, FinePICO, by combining limited annotated data of PICO entities and abundant unlabeled data. For evaluation, we divided the entire dataset into two subsets: a smaller group with annotations and a larger group without annotations. We then established the theoretical lower and upper performance bounds based on the performance of supervised learning models trained solely on the small, annotated subset and on the entire set with complete annotations, respectively. Finally, we evaluated FinePICO on both the smaller annotated subset and the larger, initially unannotated subset. We measured the performance of FinePICO using precision, recall, and F1. Results: Our method achieved precision/recall/F1 of 0.567/0.636/0.60, respectively, using a small set of annotated samples, outperforming the baseline model (F1: 0.437) by more than 16\%. The model demonstrates generalizability to a different PICO framework and to another corpus, which consistently outperforms the benchmark in diverse experimental settings (p-value \textless0.001). Conclusion: This study contributes a generalizable and effective semi-supervised approach to named entity recognition leveraging large unlabeled data together with small, annotated data. It also initially supports fine-grained PICO extraction.

Figures

Figures reproduced from arXiv: 2412.19346 by the authors.

Figure 1
Figure 1. The overview of the study workflow. 4 [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. The enhanced PICO scheme. nature and simplifies the tagging process. Additionally, we combined “outcome names” and “outcome measures” into one group to avoid redundancy and streamline the dataset. 2.6 Evaluation Metrics We tested our models on two independent test sets (PICO-Corpus, AD, and COVID-19 from Hu et al. [23]). In the first test set derived from the PICO-Corpus, we evaluated our NER models at a strict enti… view at source ↗
Figure 3
Figure 3. Performance of the proposed models using 10% annotated data augmented with in-domain, cross￾domain data. Lower bound performance is detonated as the baseline model evaluated on the test set. The upper bound refers to the baseline model trained using the whole labeled training samples and evaluated on the test set. In scenarios where limited labeled samples were available (e.g., case 1 with 10% labeled data, as shown… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: (a) Statistical performance comparison to baseline models in 6 simulated cases and (b) experi￾mental setting (in-domain, cross-domain, and all) comparison. * - p < 0.05, ** - p < 0.01, *** - p < 0.001. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 51 canonical work pages

  1. [1]

    Evidence-based medicine

    Jannette Collins. Evidence-based medicine. Journal of the American College of Radiology, 4(8): 551–554, 2007

    work page 2007

  2. [2]

    Perspective and future of evidence-based medicine

    Suning You. Perspective and future of evidence-based medicine. Stroke and vascular neurology, 1(4), 2016

    work page 2016

  3. [3]

    Principles of evidence based medicine

    Anthony K Akobeng. Principles of evidence based medicine. Archives of disease in childhood, 90(8): 837–840, 2005

    work page 2005

  4. [4]

    Ai-generated text may have a role in evidence-based medicine

    Yifan Peng, Justin F Rousseau, Edward H Shortliffe, and Chunhua Weng. Ai-generated text may have a role in evidence-based medicine. Nature medicine, 29(7):1593–1594, 2023

    work page 2023

  5. [5]

    Leveraging generative ai for clinical evidence synthesis needs to ensure trustworthiness

    Gongbo Zhang, Qiao Jin, Denis Jered McInerney, Yong Chen, Fei Wang, Curtis L Cole, Qian Yang, Yanshan Wang, Bradley A Malin, Mor Peleg, et al. Leveraging generative ai for clinical evidence synthesis needs to ensure trustworthiness. Journal of Biomedical Informatics, 153:104640, 2024. 16

    work page 2024

  6. [6]

    Meta-analysis as evidence: building a better pyramid

    Jesse A Berlin and Robert M Golub. Meta-analysis as evidence: building a better pyramid. Jama, 312 (6):603–606, 2014

    work page 2014

  7. [7]

    Systematic reviews: synthesis of best evidence for clinical decisions

    Deborah J Cook, Cynthia D Mulrow, and R Brian Haynes. Systematic reviews: synthesis of best evidence for clinical decisions. Annals of internal medicine, 126(5):376–380, 1997

    work page 1997

  8. [8]

    Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS medicine, 7(9):e1000326, 2010

    Hilda Bastian, Paul Glasziou, and Iain Chalmers. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS medicine, 7(9):e1000326, 2010

    work page 2010

Show all 59 references
  1. [9]

    Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the prospero registry

    Rohit Borah, Andrew W Brown, Patrice L Capers, and Kathryn A Kaiser. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the prospero registry. BMJ open, 7(2):e012545, 2017

    2017

  2. [10]

    Martin Dawes, Pierre Pluye, Laura Shea, Roland Grad, Arlene Greenberg, and Jian-Yun Nie. The identification of clinically important elements within medical journal abstracts: Patient–population– problem, exposure–intervention, comparison, outcome, duration and results (pecodr)...

    2007

  3. [11]

    Answering clinical questions with knowledge-based and sta- tistical techniques

    Dina Demner-Fushman and Jimmy Lin. Answering clinical questions with knowledge-based and sta- tistical techniques. Computational Linguistics, 33(1):63–103, 2007

    2007

  4. [12]

    Combination of conditional random field with a rule based method in the extraction of pico elements

    Samir Chabou and Michal Iglewski. Combination of conditional random field with a rule based method in the extraction of pico elements. BMC medical informatics and decision making, 18:1–14, 2018

    2018

  5. [13]

    Pico element detection in medical text via long short-term memory neural networks

    Di Jin and Peter Szolovits. Pico element detection in medical text via long short-term memory neural networks. In Proceedings of the BioNLP 2018 workshop, pages 67–75, 2018

    2018

  6. [14]

    Framewise phoneme classification with bidirectional lstm and other neural network architectures

    Alex Graves and J ¨urgen Schmidhuber. Framewise phoneme classification with bidirectional lstm and other neural network architectures. Neural networks, 18(5-6):602–610, 2005

    2005

  7. [15]

    End-to-end sequence labeling via bi-directional lstm-cnns-crf

    X Ma. End-to-end sequence labeling via bi-directional lstm-cnns-crf. arXiv preprint arXiv:1603.01354, 2016

    2016 arXiv

  8. [16]

    Advancing pico element detection in biomedical text via deep neural networks

    Di Jin and Peter Szolovits. Advancing pico element detection in biomedical text via deep neural networks. Bioinformatics, 36(12):3856–3862, 2020

    2020

  9. [17]

    Improving reference prioriti- sation with pico recognition

    Austin J Brockmeier, Meizhi Ju, Piotr Przybyła, and Sophia Ananiadou. Improving reference prioriti- sation with pico recognition. BMC medical informatics and decision making, 19:1–14, 2019

    2019

  10. [18]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  11. [19]

    Biobert: a pre-trained biomedical language representation model for biomedical text mining

    Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4):1234–1240, 2020

    2020

  12. [20]

    Roberta: A robustly optimized bert pretraining approach

    Yinhan Liu. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 364, 2019

    1907 arXiv

  13. [21]

    Scibert: A pretrained language model for scientific text

    Iz Beltagy, Kyle Lo, and Arman Cohan. Scibert: A pretrained language model for scientific text. arXiv preprint arXiv:1903.10676, 2019. 17

    1903 arXiv

  14. [22]

    A corpus with multi-level annotations of patients, interventions and outcomes to sup- port language processing for medical literature

    Benjamin Nye, Junyi Jessy Li, Roma Patel, Yinfei Yang, Iain J Marshall, Ani Nenkova, and By- ron C Wallace. A corpus with multi-level annotations of patients, interventions and outcomes to sup- port language processing for medical literature. In Proceedings of the conference. ...

    2018

  15. [23]

    Towards precise pico ex- traction from abstracts of randomized controlled trials using a section-specific learning approach

    Yan Hu, Vipina K Keloth, Kalpana Raja, Yong Chen, and Hua Xu. Towards precise pico ex- traction from abstracts of randomized controlled trials using a section-specific learning approach. Bioinformatics, 39(9):btad542, 2023

    2023

  16. [24]

    A study on agreement in pico span annotations

    Grace E Lee and Aixin Sun. A study on agreement in pico span annotations. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1149–1152, 2019

    2019

  17. [25]

    Correcting crowd- sourced annotations to improve detection of outcome types in evidence based medicine

    Micheal Abaho, Danushka Bollegala, Paula R Williamson, and Susanna Dodd. Correcting crowd- sourced annotations to improve detection of outcome types in evidence based medicine. In KDH@ IJCAI, pages 1–5, 2019

    2019

  18. [26]

    Not so weak pico: leveraging weak supervision for participants, interventions, and outcomes recognition for systematic review automation

    Anjani Dhrangadhariya and Henning M ¨uller. Not so weak pico: leveraging weak supervision for participants, interventions, and outcomes recognition for systematic review automation. JAMIA open, 6(1):ooac107, 2023

    2023

  19. [27]

    An annotated corpus of clinical trial publications supporting schema-based relational information extraction

    Olivia Sanchez-Graillet, Christian Witte, Frank Grimm, and Philipp Cimiano. An annotated corpus of clinical trial publications supporting schema-based relational information extraction. Journal of Biomedical Semantics, 13(1):14, 2022

    2022

  20. [28]

    Introduction to systematic review and meta-analysis

    EunJin Ahn and Hyun Kang. Introduction to systematic review and meta-analysis. Korean journal of anesthesiology, 71(2):103–112, 2018

    2018

  21. [29]

    Pico cor- pus: a publicly available corpus to support automatic data extraction from biomedical literature

    Faith Mutinda, Kongmeng Liew, Shuntaro Yada, Shoko Wakamiya, and Eiji Aramaki. Pico cor- pus: a publicly available corpus to support automatic data extraction from biomedical literature. In Proceedings of the first Workshop on Information Extraction from Scientific Publication...

    2022

  22. [30]

    Automatic text classification to support systematic reviews in medicine

    JJ Garc ´ıa Adeva, JM Pikatza Atxa, M Ubeda Carrillo, and E Ansuategi Zengotitabengoa. Automatic text classification to support systematic reviews in medicine. Expert Systems with Applications, 41 (4):1498–1508, 2014

    2014

  23. [31]

    A survey on deep semi-supervised learning

    Xiangli Yang, Zixing Song, Irwin King, and Zenglin Xu. A survey on deep semi-supervised learning. IEEE Transactions on Knowledge and Data Engineering, 35(9):8934–8954, 2022

    2022

  24. [32]

    Prototype-guided pseudo labeling for semi-supervised text classification

    Weiyi Yang, Richong Zhang, Junfan Chen, Lihong Wang, and Jaein Kim. Prototype-guided pseudo labeling for semi-supervised text classification. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (V olume1: Long Papers), pages 16369–16382, 2023

    2023

  25. [33]

    Decoupled deep neural network for semi- supervised semantic segmentation

    Seunghoon Hong, Hyeonwoo Noh, and Bohyung Han. Decoupled deep neural network for semi- supervised semantic segmentation. Advances in neural information processing systems, 28, 2015

    2015

  26. [34]

    Semi-supervised pca-based face recognition using self-training

    Fabio Roli and Gian Luca Marcialis. Semi-supervised pca-based face recognition using self-training. In Structural, Syntactic, and Statistical Pattern Recognition: Joint IAPR International Workshops, SSPR 2006 and SPR 2006, Hong Kong, China, August 17-19, 2006. Proceedings, pag...

    2006

  27. [35]

    Ecml-pkdd discovery challenge 2006 overview

    Steffen Bickel. Ecml-pkdd discovery challenge 2006 overview. In ECML-PKDD Discovery Challenge Workshop, pages 1–9, 2006

    2006

  28. [36]

    Semi-supervised document retrieval

    Ming Li, Hang Li, and Zhi-Hua Zhou. Semi-supervised document retrieval. Information Processing & Management, 45(3):341–355, 2009

    2009

  29. [37]

    Semi-supervised ranking for document retrieval

    Kevin Duh and Katrin Kirchhoff. Semi-supervised ranking for document retrieval. Computer Speech & Language, 25(2):261–281, 2011

    2011

  30. [38]

    Semi-supervised classification for extracting protein interaction sentences using dependency parsing

    Gunes Erkan, Arzucan ¨Ozg¨ur, and Dragomir Radev. Semi-supervised classification for extracting protein interaction sentences using dependency parsing. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Lan...

    2007

  31. [39]

    Domain-specific language model pretraining for biomedical natural language processing

    Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. Domain-specific language model pretraining for biomedical natural language processing. ACM Transactions on Computing for Healthcare (HEALTH), 3(1):1–23, 2021

    2021

  32. [40]

    Using pseudo-labeling to improve performance of deep neural networks for animal identification

    Rafael EP Ferreira, Yong Jae Lee, and Jo˜ao RR D´orea. Using pseudo-labeling to improve performance of deep neural networks for animal identification. Scientific Reports, 13(1):13875, 2023

    2023

  33. [41]

    Unsupervised data augmentation for consistency training

    Qizhe Xie, Zihang Dai, Eduard Hovy, Thang Luong, and Quoc Le. Unsupervised data augmentation for consistency training. Advances in neural information processing systems, 33:6256–6268, 2020

    2020

  34. [42]

    De-biasing distantly supervised named entity recognition via causal intervention

    Wenkai Zhang, Hongyu Lin, Xianpei Han, and Le Sun. De-biasing distantly supervised named entity recognition via causal intervention. arXiv preprint arXiv:2106.09233, 2021

    2021 arXiv

  35. [43]

    Improving large language models for clinical named entity recognition via prompt engineering

    Yan Hu, Qingyu Chen, Jingcheng Du, Xueqing Peng, Vipina Kuttichi Keloth, Xu Zuo, Yujia Zhou, Zehan Li, Xiaoqian Jiang, Zhiyong Lu, et al. Improving large language models for clinical named entity recognition via prompt engineering. Journal of the American Medical Informatics A...

    2024

  36. [44]

    A span-based model for extracting overlapping pico entities from randomized controlled trial publications

    Gongbo Zhang, Yiliang Zhou, Yan Hu, Hua Xu, Chunhua Weng, and Yifan Peng. A span-based model for extracting overlapping pico entities from randomized controlled trial publications. Journal of the American Medical Informatics Association, 31(5):1163–1171, 2024

    2024

  37. [45]

    Auto- matic data extraction to support meta-analysis statistical analysis: a case study on breast cancer

    Faith Wavinya Mutinda, Kongmeng Liew, Shuntaro Yada, Shoko Wakamiya, and Eiji Aramaki. Auto- matic data extraction to support meta-analysis statistical analysis: a case study on breast cancer. BMC Medical Informatics and Decision Making, 22(1):158, 2022

    2022

  38. [46]

    Natural language processing with Python: analyzing text with the natural language toolkit

    Steven Bird, Ewan Klein, and Edward Loper. Natural language processing with Python: analyzing text with the natural language toolkit. ” O’Reilly Media, Inc.”, 2009

    2009

  39. [47]

    Introduction to the conll-2000 shared task: Chunking

    Erik F Sang and Sabine Buchholz. Introduction to the conll-2000 shared task: Chunking. arXiv preprint cs/0009008, 2000

    2000 arXiv

  40. [48]

    Document information extraction via global tagging

    Shaojie He, Tianshu Wang, Yaojie Lu, Hongyu Lin, Xianpei Han, Yingfei Sun, and Le Sun. Document information extraction via global tagging. In China National Conference on Chinese Computational Linguistics, pages 145–158. Springer, 2023

    2023

  41. [49]

    Optimized glycemic control of type 2 diabetes with reinforcement learning: a proof-of-concept trial

    Guangyu Wang, Xiaohong Liu, Zhen Ying, Guoxing Yang, Zhiwei Chen, Zhiwen Liu, Min Zhang, Hongmei Yan, Yuxing Lu, Yuanxu Gao, et al. Optimized glycemic control of type 2 diabetes with reinforcement learning: a proof-of-concept trial. Nature Medicine, 29(10):2633–2642, 2023. 19

    2023

  42. [50]

    Subgroup analysis and other (mis) uses of baseline data in clinical trials

    Susan F Assmann, Stuart J Pocock, Laura E Enos, and Linda E Kasten. Subgroup analysis and other (mis) uses of baseline data in clinical trials. The Lancet, 355(9209):1064–1069, 2000

    2000

  43. [51]

    Misuse of baseline comparison tests and subgroup analyses in surgical trials

    Mohit Bhandari, PJ Devereaux, Patricia Li, Doug Mah, Ki Lim, Holger J Sch ¨unemann, and Paul Tornetta III. Misuse of baseline comparison tests and subgroup analyses in surgical trials. Clinical Orthopaedics and Related Research®, 447:247–251, 2006

    2006

  44. [52]

    seqeval: A python framework for sequence labeling evaluation

    Hiroki Nakayama. seqeval: A python framework for sequence labeling evaluation. Software available from https://github. com/chakki-works/seqeval, 2018

    2018

  45. [53]

    The automatic detection of dataset names in scientific articles

    Jenny Heddes, Pim Meerdink, Miguel Pieters, and Maarten Marx. The automatic detection of dataset names in scientific articles. Data, 6(8):84, 2021

    2021

  46. [54]

    A probabilistic model for identifying protein names and their name boundaries

    Kazuhiro Seki and Javed Mostafa. A probabilistic model for identifying protein names and their name boundaries. In Computational Systems Bioinformatics. CSB2003. Proceedings of the 2003 IEEE Bioinformatics Conference. CSB2003, pages 251–258. IEEE, 2003

    2003

  47. [55]

    Knowledge distillation for low-power object detection: A simple tech- nique and its extensions for training compact models using unlabeled data

    Amin Banitalebi-Dehkordi. Knowledge distillation for low-power object detection: A simple tech- nique and its extensions for training compact models using unlabeled data. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 769–778, 2021

    2021

  48. [56]

    Boosting semi-supervised learning by exploiting all unlabeled data

    Yuhao Chen, Xin Tan, Borui Zhao, Zhaowei Chen, Renjie Song, Jiajun Liang, and Xuequan Lu. Boosting semi-supervised learning by exploiting all unlabeled data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7548–7557, 2023

    2023

  49. [57]

    Realistic evaluation of deep semi-supervised learning algorithms

    Avital Oliver, Augustus Odena, Colin A Raffel, Ekin Dogus Cubuk, and Ian Goodfellow. Realistic evaluation of deep semi-supervised learning algorithms. Advances in neural information processing systems, 31, 2018

    2018

  50. [58]

    Unlabeled data: Now it helps, now it doesn’t

    Aarti Singh, Robert Nowak, and Jerry Zhu. Unlabeled data: Now it helps, now it doesn’t. Advances in neural information processing systems, 21, 2008

    2008

  51. [59]

    Towards more generalizable and accurate sentence classification in medical abstracts with less data

    Yan Hu, Yong Chen, and Hua Xu. Towards more generalizable and accurate sentence classification in medical abstracts with less data. Journal of Healthcare Informatics Research, 7(4):542–556, 2023. 20 Supplementary materials Supplementary Table 1: Customized Prompts for each ent...

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.