Pith. sign in

REVIEW 3 major objections 6 minor 80 references

DEUCE: Dual-diversity Enhancement and Uncertainty-awareness for Cold-start Active Learning

T0 review · 3 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DEUCE seeks to establish that the best cold-start seed set balances textual diversity, predicted-class diversity, and uncertainty, and it reports consistent accuracy gains over eight baselines across six text datasets.

desk verdict DEUCE is a well-executed empirical CSAL paper with a genuinely new dual-diversity mechanism; the main weaknesses are missing code and a few under-tested design choices, not the core claim. read the letter →

arxiv 2502.00305 v1 pith:PGN56X2F submitted 2025-02-01 cs.CL cs.AIcs.IR

Jiaxin Guo , C. L. Philip Chen , Shuzhen Li , Tong Zhang This is my paper · ORCID
classification cs.CLcs.AIcs.IR
keywords cold-startactivelearningtextclassificationdualdiversityclasstextualpredictiveuncertaintydual-neighborgraphseedsetselection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the seed set chosen before any labels exist should be diverse in two spaces at once—the space of text content and the space of predicted class labels—and should lean toward uncertain, representative examples. This matters because annotation is expensive and the first seed set shapes everything learned from it; existing cold-start active learning methods often over-sample easy, redundant texts and can neglect whole weak classes, a failure called the missed-cluster effect. DEUCE builds a dual-neighbor graph that merges the two diversity signals, propagates uncertainty through density-based clusters, and samples farthest points to form the seed set. Across six text classification datasets and three labeling budgets, the resulting seed sets consistently fine-tune to higher accuracy than eight baselines, with gains up to 2.5% on balanced datasets and 6.2% on the imbalanced TREC dataset.

What carries the argument

The central object is the Dual-Neighbor Graph (DNG), a graph that unifies two kNN graphs: one over text embeddings under cosine distance and one over calibrated label vectors under ℓ1 distance. Edges that appear in both graphs (dual-neighbor edges) receive extra weight, so traversal of the graph avoids picking points that are neighbors in text or class. The rest of the pipeline runs on this graph: density-based clustering (the HDBSCAN* algorithm) finds groups of similar documents with high predictive uncertainty, one message-passing step propagates uncertainty into each group, and farthest-point sampling from high-degree starting points generates candidate seed sets, from which the one with the highest propagated uncertainty is chosen.

What would settle it

Run DEUCE on a held-out dataset with known labels, comparing the real DEUCE to a variant whose label vectors are replaced by the true labels (oracle pseudo-labels); if the oracle variant outperforms DEUCE by a large margin, or if across the six datasets DEUCE's accuracy gain over baselines shows no relationship to the frozen PLM's zero-shot class-prediction accuracy, then the method's benefit is hostage to pseudo-label quality rather than to the dual-diversity mechanism itself.

Watch

Extended reading notes

Core claim

The paper's claim is that cold-start data selection has been neglecting an entire axis of diversity: class diversity. It argues that a seed set chosen for both textual spread and predicted-class spread, and biased toward uncertain yet representative examples, avoids the missed-cluster effect and gives the fine-tuned model a better start. The discovery is that the two diversity signals can be carried by one graph: a Dual-Neighbor Graph whose edges are shared between a kNN graph over text embeddings and a kNN graph over label vectors, where the label vectors come from transforming PLM similarities to class embeddings through the empirical distribution function. Edges shared by both graphs get extra weight. Density-based clustering then finds regions of high uncertainty, one message-passing step propagates uncertainty, and farthest-point sampling produces diverse candidates; the candidate set with the highest propagated uncertainty is annotated. The paper reports that this recipe outperforms eight cold-start active learning baselines on all six datasets and all three budgets tested, with the largest margin on the imbalanced TREC dataset.

Load-bearing premise

The load-bearing premise is that the frozen language model's predicted class labels, after calibration, are accurate enough that choosing for predicted-class diversity also chooses for true class balance; if those predictions are unreliable, both the class-diversity edges in the graph and the uncertainty ranking mislead the acquisition.

Editorial extensions

If this is right

  • Cold-start acquisition can be made class-balanced without any labeled data: predicted-class diversity from a frozen PLM serves as the signal that prevents the missed-cluster effect.
  • The largest gains come exactly where class imbalance hurts most: the imbalanced TREC dataset sees up to 6.2% improvement, and gains are generally larger at small budgets.
  • Because textual and class diversity are merged into one graph before uncertainty propagation and farthest-point sampling, diversity and informativeness influence the same selection step rather than being applied sequentially.
  • With roughly 7% artificially injected label errors, DEUCE still beats zero-shot chain-of-thought prompting in nearly all tested setups.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One unstated implication is that the same dual-neighbor construction could serve other label-free selection tasks, such as choosing demonstrations for in-context learning or building coresets for instruction tuning, wherever class names and unlabeled text are available.
  • The paper does not measure how reliable its predicted-class diversity proxy is; a natural extension is to compute agreement between the predicted-class kNN graph and the true-label kNN graph on each dataset and check whether it predicts the size of the gain.
  • The dual-edge bonus γ is fixed at 1.0, so an extension would set γ from calibration confidence, making class-diversity edges count more when pseudo-labels are trustworthy and less when they are not.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes DEUCE, a cold-start active learning (CSAL) method for text classification. DEUCE uses a frozen PLM (RoBERTa-base) to produce, for each unlabeled document, a textual embedding, a predictive embedding, and a label vector obtained by inner products with class embeddings followed by an empirical-distribution-function (e.d.f.) rank transform. One-vs-all self-information is used as an uncertainty estimate. DEUCE then builds two kNN graphs (textual space and label space), normalizes and symmetrizes them, and merges them into a Dual-Neighbor Graph (DNG) with a boost for dual-neighbor edges. HDBSCAN* identifies representatively uncertain clusters, uncertainty is propagated within clusters, and Farthest Point Sampling with multiple starting points yields candidate seed sets, from which the one with highest propagated uncertainty is chosen. The seed set is labeled and used to fine-tune RoBERTa-base. Experiments on six text classification datasets (IMDb, Yelpfull, AG's News, Yahoo! Answers, DBpedia, TREC) compare DEUCE to eight baselines at budgets 32/64/128, with 10 repetitions, and report accuracy, label imbalance (IMB), and textual diversity; additional analyses cover LLM-based serialization, zero-shot CoT, labeling noise, a random-prediction ablation, and GSM8K few-shot math reasoning. The paper claims consistent accuracy improvements, particularly on the imbalanced TREC dataset.

Significance. The paper addresses a well-motivated gap in CSAL: existing hybrid strategies treat diversity and uncertainty separately and ignore the label space. The dual-diversity idea, combining textual and predicted-label diversity in a single graph, is conceptually clean and the extensive evaluation (10 seeds, multiple budgets, multiple analyses) is a strength. The inclusion of a worst-case ablation (random predictions), a labeling-noise study, an energy/time comparison against LLM alternatives, and an out-of-domain generalization experiment (GSM8K) is commendable and goes beyond typical CSAL papers. If the load-bearing concerns about pseudo-label reliability and hyperparameter sensitivity are resolved, the method would be a solid contribution to the NLP active-learning literature.

major comments (3)
  1. [§4.6 and §5.3] The central mechanism of DEUCE relies on the predicted labels from Eq. (3) being accurate enough that diversity in the predicted-label space transfers to true class balance. However, Figure 3 reports prediction quality only for IMDb and TREC, and no numeric accuracy thresholds or overall accuracy values are given; there is no evidence for Yelpfull, AG's News, Yahoo!, or DBpedia. Moreover, the random-prediction ablation in Table 11 replaces the predictive embeddings with random vectors, which simultaneously destroys the class-diversity graph and the uncertainty signal; the large degradation on DBpedia at b=32 (from 86.0 to 63.2) could be due to either failure. I ask the authors to add an oracle-label variant (using gold labels to construct the label-space graph and uncertainty) and/or a noisy-pseudo-label experiment across all six datasets, so that the contribution of class diversity is isolated and the reliability of e.d.f. predictions is quantified.
  2. [§4.1 and Algorithm 1] The method introduces several hyperparameters that are fixed without sensitivity analysis: the number of neighbors k = 500, the HDBSCAN* minimum cluster size k_r = 3, the dual-neighbor edge boost γ = 1.0, and the number of FPS starting points (top-k, using the same k). Since the DNG and the acquisition are defined directly by these choices, the reported consistent gains could be an artifact of a particular setting. Please report ablations over reasonable ranges of k, k_r, γ, and the number of starting points, with at least one dataset and one budget, and discuss the stability of the method.
  3. [§4.2, Table 3] The main results are reported as mean ± std over 10 repetitions, but no statistical significance tests are provided. Many of the improvements are within one standard deviation (e.g., IMDb b=64: DEUCE 88.5±0.7 vs. VOTE-k 88.0±1.2; AG's News b=128: 87.5 vs. 87.0). To support the claim of 'consistently outperforms', please report paired significance tests (e.g., Wilcoxon signed-rank or bootstrap) across the 10 seeds for each setup, or provide effect sizes/confidence intervals. The same applies to the comparisons in Tables 7, 8, and 12.
minor comments (6)
  1. [Acknowledgments] The Acknowledgments section thanks 'our action editor, Sebastian Padó, and the anonymous reviewers'; this is unusual in a submitted manuscript and may breach anonymization norms. Please remove or rephrase.
  2. [Figure 3] The x-axis of Figure 3 is labeled 'Confidence Top-k' but is not defined; specify that it is the number of instances selected from most to least confident, and consider adding numeric accuracy values at representative cutoffs.
  3. [Algorithm 1] In Algorithm 1, the same symbol k is used for the number of neighbors in the kNN graphs and for the number of FPS starting points (top-k degrees). Use distinct names for these two quantities to avoid confusion.
  4. [Reproducibility] The paper does not release code or an official repository. Given the complexity of the graph normalization, fuzzy union, and HDBSCAN* steps, releasing the implementation would substantially improve reproducibility.
  5. [Tables 7, 8, 11, and 12] Tables 7, 8, 11, and 12 report point estimates without standard deviations; at minimum, state that these are single runs, or provide variance estimates for the main claims.
  6. [Section 5.2] In Section 5.2, the statement that 'gold labels may already contain around 3% errors' lacks a citation or explicit reference to Northcutt et al. (2021); please add the reference at that point.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DEUCE's seed-selection claims are tested on held-out labels, and its self-citations are peripheral.

full rationale

DEUCE is an algorithmic CSAL method, not a mathematical derivation. The seed set is produced by a fixed pipeline (PLM embeddings, e.d.f. calibration, DNG construction, HDBSCAN*, FPS) with hyperparameters k=500, kr=3, and gamma=1.0. The paper's central claim is that this seed set improves one-round fine-tuning accuracy relative to eight baselines on six datasets; that claim is evaluated by revealing the true labels of the acquired seed set and fine-tuning RoBERTa-base, with accuracy reported on held-out test sets (Section 4.2, Table 3). No parameter is fitted to those test accuracies, so the main result is not forced by construction. The e.d.f. transform makes label scores marginally uniform by definition, but this is a preprocessing choice for uncertainty estimation; it is not itself the claimed prediction, and the quality of the resulting pseudo-labels is checked against gold labels in Figure 3. The DNG weight rule favors dual-neighbor edges by design, but that defines the acquisition objective rather than predicting the evaluation metric. The random-prediction ablation (Section 5.3) removes class and uncertainty information together, which is a confound in the ablation, not a circularity. The only self-citations (Gong et al. 2019; Zhang et al. 2021; Zhang et al. 2022a) appear in the introduction and limitations as peripheral pointers and are not load-bearing for the effectiveness claim; baseline numbers are taken from the non-overlapping PATRON work (Yu et al. 2023). No self-citation chain or imported uniqueness theorem is used to justify the acquisition design. The weakest point, pseudo-label reliability, is a correctness and robustness risk rather than a circular step.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on several domain assumptions about PLM-based pseudo-labels and the validity of the graph-based heuristics. No new physical or scientific entities are introduced; the DNG is a computational construct, not an entity with independent evidence. The free parameters are user-set hyperparameters, not fit to the evaluation metric.

free parameters (4)
  • k (number of neighbors in kNN graphs) = 500
    Chosen by the authors in Section 4.1 with no sensitivity analysis; affects graph topology and FPS initial degree ranking.
  • k_r (HDBSCAN* minimum cluster size) = 3
    Set in Section 4.1 as representativeness threshold; no ablation.
  • gamma (dual-neighbor edge boost) = 1.0
    Set in Section 4.1 to distinguish E2 from E1 edges; since w_sym <= 1, gamma=1 forces dual-neighbor edges to have weight > 1, a threshold choice not ablated.
  • Number of FPS starting points (top-k degrees) = unspecified
    Algorithm 1 line 19 uses 'arg top-k deg' but k is overloaded (also neighbor count 500); the number of candidate seed sets is never stated or ablated.
assumptions (5)
  • domain assumption PLM embeddings and the prompt template capture task-related textual and predictive semantics sufficient for cold-start selection.
    Stated in Section 3.2.1; validated only indirectly via embedding correlations in Table 6 and class prediction accuracy in Figure 3.
  • domain assumption e.d.f. calibration of inner products yields meaningful pseudo-label scores for class diversity.
    Section 3.2.2; the uniformization is by construction, but the assumption that these scores align with true classes is empirically checked only on IMDb and TREC.
  • domain assumption The OVA self-information formula from Wojcik et al. (2022) is a valid uncertainty estimate for these calibrated scores.
    Section 3.2.2; the formula is imported from prior work and applied to scores that are not true probabilities.
  • domain assumption HDBSCAN* on the DNG identifies groups that are both textually and class-wise coherent, and uncertainty propagation within these groups improves selection.
    Section 3.2.4; no analysis of cluster quality or the propagation formula's sensitivity.
  • standard math The standard mathematical tools (kNN, UMAP-style normalization, fuzzy union symmetrization) behave as claimed.
    Sections 3.2.3, borrowing from McInnes et al. (2020) and Dubois and Prade (1982).

how reviews work

0 comments
Cite this review

Pith. "Pith review of DEUCE: Dual-diversity Enhancement and Uncertainty-awareness for Cold-start Active Learning." pith.science (2026). https://pith.science/paper/PGN56X2F

@misc{pith2026250200305,
  author       = {Pith},
  title        = {Pith review of: DEUCE: Dual-diversity Enhancement and Uncertainty-awareness for Cold-start Active Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PGN56X2F}},
  note         = {Machine review of arXiv:2502.00305}
}
read the original abstract

Cold-start active learning (CSAL) selects valuable instances from an unlabeled dataset for manual annotation. It provides high-quality data at a low annotation cost for label-scarce text classification. However, existing CSAL methods overlook weak classes and hard representative examples, resulting in biased learning. To address these issues, this paper proposes a novel dual-diversity enhancing and uncertainty-aware (DEUCE) framework for CSAL. Specifically, DEUCE leverages a pretrained language model (PLM) to efficiently extract textual representations, class predictions, and predictive uncertainty. Then, it constructs a Dual-Neighbor Graph (DNG) to combine information on both textual diversity and class diversity, ensuring a balanced data distribution. It further propagates uncertainty information via density-based clustering to select hard representative instances. DEUCE performs well in selecting class-balanced and hard representative data by dual-diversity and informativeness. Experiments on six NLP datasets demonstrate the superiority and efficiency of DEUCE.

Figures

Figures reproduced from arXiv: 2502.00305 by the authors.

Figure 1
Figure 1. The proposed DEUCE framework. with double [MASK] tokens: Tx := This sentence: “[X]” means [MASK]. Its [DOMAIN] is [MASK]. , where [DOMAIN] is the target domain Y, such as “sentiment”. The hidden representations of [MASK] tokens are extracted as the textual zxi and predic￾tive embeddings z yˆ|xi . They capture the intrinsic and task-related semantics. However, raw embeddings suffer from template bias and length bias … view at source ↗
Figure 2
Figure 2. The t-SNE visualization of the acquired seed set (b = 128) on IMDb dataset. Text embed￾dings are colored by their true labels. DEUCE. The t-SNE (van der Maaten and Hinton, 2008) method is used for visualization. From results in [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. The quality of class predictions with re [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

80 extracted references · 47 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Deepesh Agarwal, Pravesh Srivastava, Sergio Martin-del-Campo , Balasubramaniam Natarajan, and Babji Srinivasan. 2021. http://arxiv.org/abs/2110.03785v1 Addressing practical challenges in active learning via a hybrid query strategy . arXiv preprint arXiv:2110.03785v1

    work page Pith review arXiv 2021

  4. [4]

    Umang Aggarwal, Adrian Popescu, and Céline Hudelot. 2020. https://doi.org/10.1109/WACV45572.2020.9093475 Active learning for imbalanced datasets . In 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 1417--1426

    work page arXiv 2020

  5. [5]

    Negar Alizadeh and Fernando Castor. 2024. https://doi.org/10.1145/3644815.3644967 Green AI : A preliminary empirical study on energy consumption in DL models across different runtime infrastructures . In Proceedings of the IEEE/ACM 3rd International Conference on AI Engineering - Software Engineering for AI, CAIN '24, pages 134--139, New York, NY, USA. As...

    work page arXiv 2024

  6. [6]

    Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal

    Jordan T. Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal. 2020. https://openreview.net/forum?id=ryghZJBKPS Deep batch active learning by diverse, uncertain gradient lower bounds . In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26--30, 2020 . OpenReview.net

    work page 2020

  7. [7]

    Etienne Brangbour, Pierrick Bruneau, Thomas Tamisier, and St \'e phane Marchand-Maillet. 2022. http://arxiv.org/abs/2201.10227v1 Cold start active learning strategies in the context of imbalanced classification . arXiv preprint arXiv:2201.10227v1

    work page Pith review arXiv 2022

  8. [8]

    Ricardo J. G. B. Campello, Davoud Moulavi, and Joerg Sander. 2013. https://doi.org/10.1007/978-3-642-37456-2_14 Density-based clustering based on hierarchical density estimates . In Advances in Knowledge Discovery and Data Mining, pages 160--172, Berlin, Heidelberg. Springer Berlin Heidelberg

    doi:10.1007/978-3-642-37456-2_14 2013

Show all 80 references
  1. [9]

    Ricardo J. G. B. Campello, Davoud Moulavi, Arthur Zimek, and J\" o rg Sander. 2015. https://doi.org/10.1145/2733381 Hierarchical density estimates for data clustering, visualization, and outlier detection . ACM Transactions on Knowledge Discovery from Data, 10(1)

    2015 doi

  2. [10]

    Ernie Chang, Xiaoyu Shen, Hui-Syuan Yeh, and Vera Demberg. 2021. https://doi.org/10.18653/v1/2021.acl-short.2 On training instance selection for few-shot neural text generation . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the...

    2021 doi

  3. [11]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. https://arxiv.org/abs/2110.14168v2 Training verifiers to solve math word problems ...

    2021 arXiv

  4. [12]

    Sajib Dasgupta and Vincent Ng. 2009. https://aclanthology.org/P09-1079 Mine the easy, classify the hard: A semi-supervised approach to automatic sentiment classification . In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Jo...

    2009

  5. [13]

    Sanjoy Dasgupta. 2011. https://doi.org/10.1016/j.tcs.2010.12.054 Two faces of active learning . Theoretical Computer Science, 412(19):1767--1781. Algorithmic Learning Theory (ALT 2009)

    2011 doi

  6. [14]

    Durbin, Jennifer Doherty, Antoinette Stroup, Linda Coyle, Lynne Penberthy, and Georgia Tourassi

    Kevin De Angeli, Shang Gao, Mohammed Alawad, Hong-Jun Yoon, Noah Schaefferkoetter, Xiao-Cheng Wu, Eric B. Durbin, Jennifer Doherty, Antoinette Stroup, Linda Coyle, Lynne Penberthy, and Georgia Tourassi. 2021. https://doi.org/10.1186/s12859-021-04047-1 Deep active learning for ...

    2021 doi

  7. [15]

    Dmitriy Dligach and Martha Palmer. 2011. https://aclanthology.org/P11-2002 Good seed makes a good crop: Accelerating active learning using language modeling . In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologie...

    2011

  8. [16]

    Didier Dubois and Henri Prade. 1982. https://doi.org/10.1080/03081078208934833 A class of fuzzy measures based on triangular norms: A general framework for the combination of uncertain information . International Journal of General Systems , 8(1):43--61

    1982 doi

  9. [17]

    Liat Ein-Dor, Alon Halfon, Ariel Gera, Eyal Shnarch, Lena Dankin, Leshem Choshen, Marina Danilevsky, Ranit Aharonov, Yoav Katz, and Noam Slonim. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.638 Active learning for BERT : An empirical study . In Proceedings of the 2020 Con...

    2020 doi

  10. [18]

    Anton Eklund and Mona Forsman. 2022. https://doi.org/10.18653/v1/2022.emnlp-industry.65 Topic modeling by clustering language model embeddings: Human validation on an industry dataset . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: ...

    2022 doi

  11. [19]

    Yuval Eldar, Micahel Lindenbaum, Moshe Porat, and Yehoshua Y. Zeevi. 1994. https://doi.org/10.1109/ICPR.1994.577129 The farthest point strategy for progressive image sampling . In Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 2 - Conference...

    1994

  12. [20]

    Yaron Fairstein, Oren Kalinsky, Zohar Karnin, Guy Kushilevitz, Alexander Libov, and Sofia Tolmach. 2024. https://aclanthology.org/2024.law-1.8 Class balancing for efficient active learning in imbalanced datasets . In Proceedings of The 18th Linguistic Annotation Workshop (LAW-...

    2024

  13. [21]

    Jun Gao, Di He, Xu Tan, Tao Qin, Liwei Wang, and Tieyan Liu. 2019. https://openreview.net/forum?id=SkEYojRqtm Representation degeneration problem in training natural language generation models . In 7th International Conference on Learning Representations, ICLR 2019, New Orlean...

    2019

  14. [22]

    Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.552 S im CSE : Simple contrastive learning of sentence embeddings . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894--6910, Online ...

    2021 doi

  15. [23]

    Xin-Rong Gong, Jian-Xiu Jin, and Tong Zhang. 2019. https://doi.org/10.1109/BIBM47256.2019.8983025 Sentiment analysis using autoregressive language modeling and broad learning system . In 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 1130--1134

    2019

  16. [24]

    Guy Hacohen, Avihu Dekel, and Daphna Weinshall. 2022. https://proceedings.mlr.press/v162/hacohen22a.html Active learning on a budget: Opposite strategies suit high and low budgets . In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedi...

    2022

  17. [25]

    Stefan Hegselmann, Alejandro Buendia, Hunter Lang, Monica Agrawal, Xiaoyi Jiang, and David Sontag. 2023. https://proceedings.mlr.press/v206/hegselmann23a.html TabLLM : Few-shot classification of tabular data with large language models . In Proceedings of The 26th International...

    2023

  18. [26]

    Marek Herde, Denis Huseljic, Bernhard Sick, and Adrian Calma. 2021. https://doi.org/10.1109/ACCESS.2021.3135514 A survey on cost types, interaction schemes, and annotator performance models in selection algorithms for active learning in classification . IEEE Access, 9:166970--166989

    2021

  19. [27]

    Andreas Holzinger. 2016. https://doi.org/10.1007/s40708-016-0042-6 Interactive machine learning for health informatics: When do we need the human-in-the-loop? Brain Informatics, 3(2):119--131

    2016 doi

  20. [28]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 L o RA : Low-rank adaptation of large language models . In The Tenth International Conference on Learning Represen...

    2022

  21. [29]

    Rong Hu, Brian Mac Namee, and Sarah Jane Delany. 2010. http://www.aaai.org/ocs/index.php/FLAIRS/2010/paper/view/1305 Off to a good start: Using clustering to select the initial training set in active learning . In Proceedings of the Twenty-Third International Florida Artificia...

    2010

  22. [30]

    Ting Jiang, Shaohan Huang, Zhongzhi Luan, Deqing Wang, and Fuzhen Zhuang. 2023. http://arxiv.org/abs/2307.16645v1 Scaling sentence embeddings with large language models . arXiv preprint arXiv:2307.16645v1

    2023 arXiv

  23. [31]

    Ting Jiang, Jian Jiao, Shaohan Huang, Zihan Zhang, Deqing Wang, Fuzhen Zhuang, Furu Wei, Haizhen Huang, Denvy Deng, and Qi Zhang. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.603 P rompt BERT : Improving BERT sentence embeddings with prompts . In Proceedings of the 2022 C...

    2022 doi

  24. [32]

    Zhengbao Jiang, Jun Araki, Haibo Ding, and Graham Neubig. 2021. https://doi.org/10.1162/tacl_a_00407 How can we know when language models know? O n the calibration of language models for question answering . Transactions of the Association for Computational Linguistics, 9:962--977

    2021 doi

  25. [33]

    Jaeho Kang, Kwang Ryel Ryu, and Hyuk-Chul Kwon. 2004. https://doi.org/10.1007/978-3-540-24775-3_46 Using cluster-based sampling to select initial training set for active learning in text classification . In Advances in Knowledge Discovery and Data Mining, pages 384--388, Berli...

    2004 doi

  26. [34]

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. https://proceedings.neurips.cc/paper_files/paper/2022/file/8bb0d291acd4acf06ef112099c16f326-Paper-Conference.pdf Large language models are zero-shot reasoners . In Advances in Neural Infor...

    2022

  27. [35]

    Ahuja, Mahesh Subedar, Omesh Tickoo, and Ravi R

    Ranganath Krishnan, Alok Sinha, Nilesh A. Ahuja, Mahesh Subedar, Omesh Tickoo, and Ravi R. Iyer. 2021. https://icml.cc/virtual/2021/13299 Mitigating sampling bias and improving robustness in active learning . In Proceedings of Workshop on Human in the Loop Learning (HILL) in I...

    2021

  28. [36]

    Mendes, Sebastian Hellmann, Mohamed Morsey, Patrick van Kleef, S\" o ren Auer, and Christian Bizer

    Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, Dimitris Kontokostas, Pablo N. Mendes, Sebastian Hellmann, Mohamed Morsey, Patrick van Kleef, S\" o ren Auer, and Christian Bizer. 2015. https://doi.org/10.3233/sw-140134 DB pedia -- a large-scale, multilingual knowledge ba...

    2015 doi

  29. [37]

    Xin Li and Dan Roth. 2002. https://aclanthology.org/C02-1150 Learning question classifiers . In COLING 2002: The 19th International Conference on Computational Linguistics

    2002

  30. [38]

    Yansong Li, Zhixing Tan, and Yang Liu. 2023. http://arxiv.org/abs/2305.06212v1 Privacy-preserving prompt tuning for large language model services . arXiv preprint arXiv:2305.06212v1

    2023

  31. [39]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. http://arxiv.org/abs/1907.11692v1 RoBERTa : A robustly optimized BERT pretraining approach . arXiv preprint arXiv:1907.11692v1

    2019 arXiv

  32. [40]

    Yuxuan Lu, Bingsheng Yao, Shao Zhang, Yun Wang, Peng Zhang, Tun Lu, Toby Jia-Jun Li, and Dakuo Wang. 2023. http://arxiv.org/abs/2311.09825v1 Human still wins over LLM : An empirical study of active learning on domain-specific annotation tasks . arXiv preprint arXiv:2311.09825v1

    2023 arXiv

  33. [41]

    Maas, Raymond E

    Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. https://aclanthology.org/P11-1015 Learning word vectors for sentiment analysis . In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Hu...

    2011

  34. [42]

    Laurens van der Maaten and Geoffrey Hinton. 2008. http://jmlr.org/papers/v9/vandermaaten08a.html Visualizing data using t- SNE . Journal of Machine Learning Research, 9(86):2579--2605

    2008

  35. [43]

    Diego Marcheggiani and Thierry Arti \`e res. 2014. https://doi.org/10.3115/v1/D14-1097 An experimental comparison of active learning strategies for partially labeled sequences . In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing ( EMNLP )...

    2014 doi

  36. [44]

    Katerina Margatina, Giorgos Vernikos, Lo \" c Barrault, and Nikolaos Aletras. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.51 Active learning by acquiring contrastive examples . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pag...

    2021 doi

  37. [45]

    Leland McInnes, John Healy, and James Melville. 2020. http://arxiv.org/abs/1802.03426v3 UMAP : Uniform manifold approximation and projection for dimension reduction . arXiv preprint arXiv:1802.03426v3

    2020 arXiv

  38. [46]

    Leland McInnes, John Healy, Nathaniel Saul, and Lukas Großberger. 2018. https://doi.org/10.21105/joss.00861 UMAP : Uniform manifold approximation and projection . Journal of Open Source Software, 3(29):861

    2018 doi

  39. [47]

    Yu Meng, Jiaming Shen, Chao Zhang, and Jiawei Han. 2019. https://doi.org/10.1609/aaai.v33i01.33016826 Weakly-supervised hierarchical text classification . Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):6826--6833

    2019 doi

  40. [48]

    Pu Miao, Zeyao Du, and Junlin Zhang. 2023. https://doi.org/10.1145/3583780.3614833 DebCSE : Rethinking unsupervised contrastive sentence embedding learning in the debiasing perspective . In Proceedings of the 32nd ACM International Conference on Information and Knowledge Manag...

    2023

  41. [49]

    Thomas M\" u ller, Guillermo P\' e rez-Torr\' o , Angelo Basile, and Marc Franco-Salvador. 2022. https://doi.org/10.1007/978-3-031-08473-7_9 Active few-shot learning with FASL . In Natural Language Processing and Information Systems; 27th International Conference on Applicatio...

    2022 doi

  42. [50]

    Saeid Alavi Naeini, Raeid Saqur, Mozhgan Saeidi, John Michael Giorgi, and Babak Taati. 2023. https://openreview.net/forum?id=ZV4tZgclu8 Large language models are fixated by red herrings: Exploring creative problem solving and E instellung effect using the O nly C onnect W all ...

    2023

  43. [51]

    Nguyen and Arnold Smeulders

    Hieu T. Nguyen and Arnold Smeulders. 2004. https://doi.org/10.1145/1015330.1015349 Active learning using pre-clustering . In Proceedings of the Twenty-First International Conference on Machine Learning, ICML '04, page 79, New York, NY, USA. Association for Computing Machinery

    2004

  44. [52]

    Curtis Northcutt, Anish Athalye, and Jonas Mueller. 2021. https://datasets-benchmarks-proceedings.neurips.cc/paper_files/paper/2021/file/f2217062e9a397a1dca429e7d70bc6ca-Paper-round1.pdf Pervasive label errors in test sets destabilize machine learning benchmarks . In Proceedin...

    2021

  45. [53]

    Shreyas Padhy, Zachary Nado, Jie Ren, Jeremiah Liu, Jasper Snoek, and Balaji Lakshminarayanan. 2020. https://www.gatsby.ucl.ac.uk/ balaji/udl2020/accepted-papers/UDL2020-paper-040.pdf Revisiting one-vs-all classifiers for predictive uncertainty and out-of-distribution detectio...

    2020

  46. [54]

    Shirui Pan, Linhao Luo, Yufei Wang, Chen Chen, Jiapu Wang, and Xindong Wu. 2024. https://doi.org/10.1109/TKDE.2024.3352100 Unifying large language models and knowledge graphs: A roadmap . IEEE Transactions on Knowledge and Data Engineering, pages 1--20

    2024

  47. [55]

    Seo Yeon Park and Cornelia Caragea. 2022. https://doi.org/10.18653/v1/2022.acl-long.368 On the calibration of pre-trained language models using mixup guided by area under the margin and saliency . In Proceedings of the 60th Annual Meeting of the Association for Computational L...

    2022 doi

  48. [56]

    Rudolph, Nicholas T

    Kara E. Rudolph, Nicholas T. Williams, Caleb H. Miles, Joseph Antonelli, and Ivan Diaz. 2023. https://doi.org/doi:10.1515/jci-2023-0022 All models are wrong, but which are useful? C omparing parametric and nonparametric estimation of causal effects in finite samples . Journal ...

    2023 doi

  49. [57]

    Christopher Schr \"o der, Andreas Niekler, and Martin Potthast. 2022. https://doi.org/10.18653/v1/2022.findings-acl.172 Revisiting uncertainty-based query strategies for active learning with transformers . In Findings of the Association for Computational Linguistics: ACL 2022,...

    2022 doi

  50. [58]

    Pedersen

    Hinrich Sch\" u tze, Emre Velipasaoglu, and Jan O. Pedersen. 2006. https://doi.org/10.1145/1183614.1183709 Performance thresholding in practical text classification . In Proceedings of the 15th ACM International Conference on Information and Knowledge Management, CIKM '06, pag...

    2006

  51. [59]

    Ozan Sener and Silvio Savarese. 2018. https://openreview.net/forum?id=H1aIuk-RW Active learning for convolutional neural networks: A core-set approach . In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 -- May 3, 2018, Conf...

    2018

  52. [60]

    Sentence Transformers . 2024. https://doi.org/10.57967/hf/2004 paraphrase-mpnet-base-v2 (revision e6981e5 ) . Hugging Face

    2024 doi

  53. [61]

    Burr Settles. 2009. https://research.cs.wisc.edu/techreports/2009/TR1648.pdf Active learning literature survey . Computer Sciences Technical Report 1648, University of Wisconsin--Madison

    2009

  54. [62]

    Eyal Shnarch, Ariel Gera, Alon Halfon, Lena Dankin, Leshem Choshen, Ranit Aharonov, and Noam Slonim. 2022. https://doi.org/10.18653/v1/2022.acl-long.526 Cluster & tune: B oost cold start performance in text classification . In Proceedings of the 60th Annual Meeting of the Asso...

    2022 doi

  55. [63]

    Smith, and Tao Yu

    Hongjin Su, Jungo Kasai, Chen Henry Wu, Weijia Shi, Tianlu Wang, Jiayi Xin, Rui Zhang, Mari Ostendorf, Luke Zettlemoyer, Noah A. Smith, and Tao Yu. 2023. https://openreview.net/forum?id=qY1hlv7gwg Selective annotation makes language models better few-shot learners . In The Ele...

    2023

  56. [64]

    Katrin Tomanek, Florian Laws, Udo Hahn, and Hinrich Sch \"u tze. 2009. https://aclanthology.org/W09-1902 On proper unit selection in active learning: Co-selection effects for named entity recognition . In Proceedings of the NAACL HLT 2009 Workshop on Active Learning for Natura...

    2009

  57. [65]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, W...

    2023 arXiv

  58. [66]

    Cheng Wang. 2024. http://arxiv.org/abs/2308.01222v2 Calibration in deep learning: A survey of the state-of-the-art . arXiv preprint arXiv:2308.01222v2

    2024 arXiv

  59. [67]

    Le, and Denny Zhou

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V. Le, and Denny Zhou. 2022. https://proceedings.neurips.cc/paper_files/paper/2022/file/9d5609613524ecf4f15af0f7b31abca4-Paper-Conference.pdf Chain-of-thought prompting elicits reasonin...

    2022

  60. [68]

    Xingjiao Wu, Luwei Xiao, Yixuan Sun, Junhang Zhang, Tianlong Ma, and Liang He. 2022. https://doi.org/10.1016/j.future.2022.05.014 A survey of human-in-the-loop for machine learning . Future Generation Computer Systems, 135:364--381

    2022 doi

  61. [69]

    Bartosz Wójcik, Jacek Grela, Marek Śmieja, Krzysztof Misztal, and Jacek Tabor. 2022. https://doi.org/10.1016/j.asoc.2022.109219 SLOVA : Uncertainty estimation using single label one-vs-all classifier . Applied Soft Computing, 126:109219

    2022

  62. [70]

    Salakhutdinov, and Quoc V

    Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R. Salakhutdinov, and Quoc V. Le. 2019. https://proceedings.neurips.cc/paper_files/paper/2019/file/dc6a7e655d7e5840e66733e9ee67cc69-Paper.pdf XLN et: Generalized autoregressive pretraining for language understanding ....

    2019

  63. [71]

    Hualong Yu, Xibei Yang, Shang Zheng, and Changyin Sun. 2019. https://doi.org/10.1109/TNNLS.2018.2855446 Active learning from imbalanced data: A solution of online weighted extreme learning machine . IEEE Transactions on Neural Networks and Learning Systems, 30(4):1088--1103

    2019

  64. [72]

    Yue Yu, Rongzhi Zhang, Ran Xu, Jieyu Zhang, Jiaming Shen, and Chao Zhang. 2023. https://doi.org/10.18653/v1/2023.acl-long.141 Cold-start data selection for better few-shot language model fine-tuning: A prompt-based uncertainty propagation approach . In Proceedings of the 61st ...

    2023 doi

  65. [73]

    Michelle Yuan, Hsuan-Tien Lin, and Jordan Boyd-Graber. 2020 a . https://doi.org/10.18653/v1/2020.emnlp-main.637 Cold-start active learning through self-supervised language modeling . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMN...

    2020 doi

  66. [74]

    Mu Yuan, Lan Zhang, Xiang-Yang Li, and Hui Xiong. 2020 b . https://doi.org/10.1109/ICDE48307.2020.00188 Comprehensive and efficient data labeling via adaptive model scheduling . In 2020 IEEE 36th International Conference on Data Engineering (ICDE), pages 1858--1861

    2020

  67. [75]

    Shiwei Zhang, Mingfang Wu, and Xiuzhen Zhang. 2023. http://arxiv.org/abs/2310.11318v1 Utilising a large language model to annotate subject metadata: A case study in an A ustralian national research data catalogue . arXiv preprint arXiv:2310.11318v1

    2023 arXiv

  68. [76]

    Tong Zhang, Xinrong Gong, and C. L. Philip Chen. 2022 a . https://doi.org/10.1109/TCYB.2021.3050508 BMT-N et: Broad multitask transformer network for sentiment analysis . IEEE Transactions on Cybernetics, 52(7):6232--6243

    2022

  69. [77]

    Tong Zhang, Guoxi Su, Chunmei Qing, Xiangmin Xu, Bolun Cai, and Xiaofen Xing. 2021. https://doi.org/10.1109/TSMC.2018.2884996 Hierarchical lifelong learning by sharing representations and integrating hypothesis . IEEE Transactions on Systems, Man, and Cybernetics: Systems, 51(...

    2021

  70. [78]

    Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. https://proceedings.neurips.cc/paper_files/paper/2015/file/250cf8b51c773f3f8dc8b4be867a9a02-Paper.pdf Character-level convolutional networks for text classification . In Advances in Neural Information Processing Systems, volume 28...

    2015

  71. [79]

    Zhisong Zhang, Emma Strubell, and Eduard Hovy. 2022 b . https://doi.org/10.18653/v1/2022.emnlp-main.414 A survey of active learning for natural language processing . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6166--6190, Ab...

    2022 doi

  72. [80]

    Jingbo Zhu, Huizhen Wang, Tianshun Yao, and Benjamin K. Tsou. 2008. https://aclanthology.org/C08-1143 Active learning with sampling by uncertainty and density for word sense disambiguation and text classification . In Proceedings of the 22nd International Conference on Computa...

    2008

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.