Pith. sign in

REVIEW 4 major objections 5 minor 68 references

Your Pretrained Model Tells the Difficulty Itself: A Self-Adaptive Curriculum Learning Paradigm for Natural Language Understanding

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a pretrained model's own prompt-based confidence margin, read at a masked token before fine-tuning, is a reliable difficulty score whose use as a training-order signal yields faster convergence and equal-or-better…

desk verdict The paper's central claim is undercut by its own Table 1; the idea is cheap and clearly described, but the evidence doesn't support the abstract. read the letter →

arxiv 2507.09758 v1 pith:PNRHJII6 submitted 2025-07-13 cs.CL cs.LG

Qi Feng , Yihong Liu , Hinrich Schütze This is my paper
classification cs.CLcs.LG
keywords curriculumlearningself-adaptivedifficultyscoreprompt-basedfine-tuningmaskedlanguagemodelingsamplingstrategiesnaturalunderstandingfew-shot
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a pretrained language model can grade its own fine-tuning data: fed a prompt with a masked word, the model's confidence in choosing among verbalizer keywords gives each example a difficulty score, and ordering or sampling the training set by that score improves fine-tuning. The motivation is that standard difficulty heuristics such as sentence length do not capture what a particular model actually finds hard. Experiments on sentiment analysis, hate speech detection, and natural language inference compare six ordering strategies against random and length-based baselines, reporting faster convergence during a single epoch and equal-or-better final accuracy in the full-data setting, with stronger gains on fine-grained and imbalanced datasets. If the claim holds, curriculum learning becomes self-adaptive in the sense that the model supplies its own curriculum at no extra parameter cost.

What carries the argument

The load-bearing object is the prompt-based difficulty score: for each example, the model fills a [MASK] token in a hand-written template, the softmax probability of that token is mapped to class labels through a verbalizer, and the score is the absolute difference between the top two verbalizer class probabilities, $|P_{\mathrm{pos}} - P_{\mathrm{neg}}|$ for binary and $|P_{\max} - P_{\text{second-max}}|$ for multiclass tasks. This one number, computed once before training with no parameter updates, ranks every example and feeds all six sampling strategies; the paper's empirical case rests on that rank ordering capturing the model's own uncertainty better than surface heuristics do.

What would settle it

Recompute pre-training difficulty scores on SST-2 with at least two alternative prompt templates and verbalizer keyword sets; if the example rankings change enough that the best strategy under one prompt scores no better than random under another, the claim that the model 'tells the difficulty itself' fails.

Watch

Extended reading notes

Core claim

The central discovery is that the margin between the two highest class probabilities obtained from a frozen pretrained model's masked-token prediction, mapped through a verbalizer, is a usable proxy for example difficulty in NLU fine-tuning. Examples with small margins are exactly the ones the model tends to misclassify, both before and after training. The paper then turns this single precomputed score into a family of six sampling strategies spanning naive easy-to-hard and hard-to-easy orders, rank-based probabilistic sampling, and partitioned batches that mix easy and hard samples with a 6:4 emphasis; across BERT and RoBERTa on SST-2, SST-5, HSOL, and XNLI, the strategies that prioritize difficult examples, especially partitioned-batch PMD, give the most consistent improvements over random sampling, and most strategies beat both baselines in a 64-example few-shot setting.

Load-bearing premise

The method stands on the premise that how confidently a frozen pretrained model fills a masked word in a hand-written prompt, before any fine-tuning, tells you which examples are genuinely hard for that model, and that this single static ranking remains useful across all five epochs of training.

Editorial extensions

If this is right

  • Model-predicted difficulty can replace hand-built metrics like sentence length or word rarity as the basis for curriculum learning, so curricula become model-specific rather than dataset-generic.
  • The best strategy is not easy-to-hard: partitioned batches that emphasize difficult examples (PMD) deliver the most consistent gains, suggesting hard examples are worth prioritizing during fine-tuning.
  • In the 64-example few-shot setting, most proposed strategies beat random and length baselines, with hard-example-first strategies especially valuable, so the same scoring can guide data selection when data are scarce.
  • Because scoring costs only one forward pass per example and no extra parameters, the approach scales to full datasets and could be combined with other fine-tuning regimes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The scores are computed once on the frozen pretrained model and never updated, even though the title says 'self-adaptive'; a natural extension the paper does not test is re-scoring after each epoch and letting the curriculum track the model's changing competence.
  • Since the paper concedes that prompt-based learning is highly sensitive to prompt design, the difficulty ranking itself may shift with template or verbalizer wording; automated verbalizer search over several templates would reveal how much of the gain is intrinsic to the model versus the chosen prompt.
  • The margin-based difficulty score is essentially an uncertainty estimate, so the paradigm connects to active learning and data pruning: the same score could select which examples to label or which to omit, not just which order to train in.
  • On imbalanced datasets like HSOL, focusing sampling on difficult examples may partly act as class rebalancing, since minority-class examples are likely to receive low confidence scores; a test on perfectly balanced subsets would separate class-balance effects from difficulty effects.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a self-adaptive curriculum learning paradigm in which the difficulty of each fine-tuning example is estimated by a frozen pretrained language model's confidence in a cloze-style prompt, defined as the margin between the top two verbalizer class probabilities (§3.3). Six sampling strategies are introduced: naive sequential easy-to-hard (E2D) and hard-to-easy (D2E), probability-based sampling (SME, SMD), and partitioned batch sampling (PME, PMD) (§3.4). The method is evaluated on SST-2, SST-5, HSOL, and XNLI with BERT and RoBERTa in both full-dataset and 64-example few-shot settings (Tables 1 and 2), plus one-epoch checkpoint analyses (Figures 4–5 and Appendix C). The abstract claims faster convergence and improved performance relative to standard random sampling.

Significance. If the empirical claims held, the paper would offer a cheap, parameter-free difficulty signal that requires no external model or annotation, and the systematic comparison of six curriculum strategies on four NLU tasks is a reasonable design. The paper makes code available and covers two backbone models, which are useful strengths. However, the central empirical claim is contradicted by the paper's own Table 1: only 11 of 48 full-dataset accuracy comparisons exceed Random, 35 are below, and 2 tie; all six proposed strategies underperform Random for RoBERTa on SST-2 and for BERT on XNLI. Without standard deviations, confidence intervals, or significance tests, the few positive differences are not interpretable. As presented, the contribution is therefore not supported by the evidence.

major comments (4)
  1. [Abstract; Table 1] The headline claim of 'improved performance compared to standard random sampling' is not supported by the paper's own results. In Table 1, counting only the Acc column in the full-dataset setting (4 datasets × 2 backbones × 6 strategies = 48 comparisons), 11 of the proposed-strategy cells exceed Random, 35 are below, and 2 tie. The failures include complete blocks: all six strategies underperform Random for RoBERTa on SST-2 (Random 94.11 vs 93.27–93.92) and for BERT on XNLI (Random 84.01 vs 82.12–83.75). Results are means over three seeds with no standard deviations or significance tests, so the few positive differences of about 0.2–1.1 accuracy points cannot be distinguished from noise. The few-shot table (Table 2) is likewise mixed; for example, on RoBERTa HSOL both PME (80.64) and PMD (79.99) are below Random (80.67). This directly undermines the central claim.
  2. [§5.1] The prose in §5.1 misreports the table. The statement that 'PMD achieves the highest performance in most cases' is not borne out: by accuracy, PMD is best in only three of the eight model–dataset blocks (BERT SST-2, RoBERTa HSOL, RoBERTa XNLI) and is the worst proposed strategy on RoBERTa SST-2 (93.27 vs Random 94.11). The statement that 'The baseline Length achieves slightly better performance than Random' is contradicted by Table 1: Length beats Random in only one of eight blocks (BERT SST-2) and is lower than Random in the other seven.
  3. [§5.2; Appendix A.5] The 'faster convergence' component of the abstract is not established. The checkpoint curves in Figures 4–5 and Appendix C are from a single seed (seed 66, as stated in A.5), have no error bars, and cover only one epoch. Figure 4 itself shows that at the final checkpoint 'most methods, including the baselines, continue to improve', and the curves fluctuate substantially. No quantitative convergence measure (e.g., time to a target accuracy or area under the curve) is reported, so even a descriptive advantage in early checkpoints does not support the stated faster-convergence claim.
  4. [§3.3; Limitations; Appendix A] The difficulty signal rests on hand-chosen prompt templates and single-keyword verbalizers (one per dataset, given in Appendix A), and the Limitations section concedes that prompt-based learning is 'highly sensitive to prompt design'. No ablation over templates or verbalizer keywords is reported, so the robustness of the scores that drive all six strategies to these free parameters is unknown. This is not circular reasoning—using the model's own confidence is a legitimate design choice—but it is a load-bearing sensitivity concern for the proposed mechanism.
minor comments (5)
  1. [Figure 1] The caption of Figure 1 does not name the dataset, model, or training stage beyond 'before and after 1 epoch'; please specify these, since the figure is the principal validation that the difficulty score correlates with errors.
  2. [§3.3] The softmax equation in §3.3 is garbled ('P (wi) = eziP wj ∈V ezj'); it should be typeset as P(w_i)=e^{z_i}/Σ_{w_j∈V} e^{z_j}.
  3. [References] The reference list is duplicated: the same list of references appears twice after Section 6. It should appear once.
  4. [Tables 1–2] The red/blue/white background coding in Tables 1 and 2 is never explained in the text or caption and will be invisible in grayscale; please add a legend or textual markers.
  5. [A.5] The partition ratio |B1|:|B2| = 6:4 is said to be 'based on preliminary results' with no details; please document the preliminary experiments or state that this is a fixed hyperparameter.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claim rests on external baseline comparisons, and the self-citations are contextual rather than load-bearing.

full rationale

The paper's central claim is empirical: difficulty scores computed from a frozen pretrained model's prompt-based class-probability margin are used to rank fine-tuning examples, and the resulting curricula are compared against Random and Length baselines in Tables 1 and 2. The difficulty score in Section 3.3 is a design choice rather than a fitted parameter renamed as a prediction; it is computed once from the pretrained model, and the ranking is static. The validation in Figure 1, showing that low-score examples are more often misclassified, is an empirical calibration property and not an identity: for binary classification a high margin can still be wrong when the model is confidently incorrect, so the correlation between margin and accuracy is not definitional. The self-citations, such as Schick and Schütze (2021a) for prompt and verbalizer methodology and Ma et al. (2023) for single-keyword verbalizers, are contextual and are not used to justify the central performance claim; that claim rests on the paper's own experiments against external baselines. The Limitations section concedes that prompt-based learning is highly sensitive to prompt design, but that is a robustness concern rather than a circular step. Even if the abstract's claim of improved performance over random sampling is not fully supported by the numbers in Table 1, that is an evidentiary weakness, not circularity. No load-bearing equation reduces to its own input, and no prediction is statistically forced by a fit.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

No new particles, forces, or mathematical objects are introduced; the 'difficulty score' is a computed statistic. The central empirical claims rest instead on hand-chosen design elements (prompts, verbalizers, a 6:4 partition ratio) and on the assumed validity of static self-confidence as a difficulty signal.

free parameters (5)
  • Partition ratio |B1|:|B2| = 6:4
    Set in §3.4.3 'based on preliminary results'; used only by PME and PMD.
  • Prompt templates = Per-task templates in Appendix A (e.g., 'this was a [MASK] movie.')
    Manually designed for each dataset; difficulty scores are sensitive to these choices, as the Limitations section acknowledges.
  • Verbalizer keywords = Single tokens per class (e.g., great/bad, hateful/offensive/neutral)
    Hand-selected in §3.2 and Appendix A; authors state single keywords are optimal based on preliminary results.
  • Learning rate = 1e-5
    Tuned in Appendix A.5 for optimal performance.
  • Number of epochs = 5 (main), 1 (checkpoint analysis)
    Fixed in A.5 for consistency across strategies.
assumptions (4)
  • domain assumption A pretrained MLM's token probabilities at [MASK], mapped through a verbalizer, reflect the model's confidence per example.
    Core of §3.3; validated only by Figure 1, whose dataset is unspecified.
  • domain assumption The margin between top-two class probabilities is a valid measure of example difficulty.
    Defined in §3.3; no theoretical grounding or human agreement check (the latter is noted as a limitation).
  • domain assumption Difficulty scores computed once from the frozen pretrained model remain a useful training-order signal after the model changes through fine-tuning.
    Scores are static, yet Figure 1 shows the model changes substantially over one epoch.
  • domain assumption Standard classification fine-tuning (AdamW, linear schedule, no warm-up) behaves as expected for these PLMs.
    Background premise of all experiments; not proved in the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Your Pretrained Model Tells the Difficulty Itself: A Self-Adaptive Curriculum Learning Paradigm for Natural Language Understanding." pith.science (2026). https://pith.science/paper/PNRHJII6

@misc{pith2026250709758,
  author       = {Pith},
  title        = {Pith review of: Your Pretrained Model Tells the Difficulty Itself: A Self-Adaptive Curriculum Learning Paradigm for Natural Language Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PNRHJII6}},
  note         = {Machine review of arXiv:2507.09758}
}
read the original abstract

Curriculum learning is a widely adopted training strategy in natural language processing (NLP), where models are exposed to examples organized by increasing difficulty to enhance learning efficiency and performance. However, most existing approaches rely on manually defined difficulty metrics -- such as text length -- which may not accurately reflect the model's own perspective. To overcome this limitation, we present a self-adaptive curriculum learning paradigm that prioritizes fine-tuning examples based on difficulty scores predicted by pre-trained language models (PLMs) themselves. Building on these scores, we explore various training strategies that differ in the ordering of examples for the fine-tuning: from easy-to-hard, hard-to-easy, to mixed sampling. We evaluate our method on four natural language understanding (NLU) datasets covering both binary and multi-class classification tasks. Experimental results show that our approach leads to faster convergence and improved performance compared to standard random sampling.

Figures

Figures reproduced from arXiv: 2507.09758 by the authors.

Figure 1
Figure 1. Frequencies of samples being incorrectly [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the proposed difficulty scor [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. An illustration of our sampling strategies. Each example is associated with a difficulty score based on the [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (16 more)
Figure 4
Figure 4. Figure 4: Progression of accuracy during a single epoch [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Progression of accuracy during a single epoch [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Average evaluation accuracy on BERT recorded at 10 checkpoints during a single epoch on SST-2. 2 4 6 8 10 Checkpoint 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 Avg Eval Loss BERT on SST-2: Evaluation Loss Over Checkpoints Random Length E2D D2E SME SMD PME PMD [PITH_…
Figure 7
Figure 7. Figure 7: Average evaluation loss on BERT recorded at [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Average evaluation accuracy on BERT recorded at 10 checkpoints during a single epoch on SST-5. 2 4 6 8 10 Checkpoint 1.10 1.15 1.20 1.25 1.30 1.35 1.40 Avg Eval Loss BERT on SST-5: Evaluation Loss Over Checkpoints Random Length E2D D2E SME SMD PME PMD [PITH_FULL_IMAGE…
Figure 9
Figure 9. Figure 9: Average evaluation loss on BERT recorded at [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 13
Figure 13. Figure 13: Average evaluation loss on BERT recorded [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]
Figure 10
Figure 10. Figure 10: Average evaluation accuracy on BERT recorded at 10 checkpoints during a single epoch on HSOL. 2 4 6 8 10 Checkpoint 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 Avg Eval Loss BERT on HSOL: Evaluation Loss Over Checkpoints Random Length E2D D2E SME SMD PME PMD [PITH_FULL_I…
Figure 14
Figure 14. Figure 14: Average evaluation loss on RoBERTa recorded at 10 checkpoints during a single epoch on SST-2. From [PITH_FULL_IMAGE:figures/full_fig_p019_14.png]
Figure 15
Figure 15. Figure 15: Average evaluation loss on RoBERTa recorded at 10 checkpoints during a single epoch on SST-5. As shown in [PITH_FULL_IMAGE:figures/full_fig_p019_15.png]
Figure 16
Figure 16. Figure 16: Average evaluation accuracy on RoBERTa recorded at 10 checkpoints during a single epoch on HSOL [PITH_FULL_IMAGE:figures/full_fig_p020_16.png]
Figure 17
Figure 17. Figure 17: Average evaluation loss on RoBERTa recorded at 10 checkpoints during a single epoch on HSOL. According to [PITH_FULL_IMAGE:figures/full_fig_p020_17.png]
Figure 20
Figure 20. Figure 20: Sample difficulty score distributions on SST [PITH_FULL_IMAGE:figures/full_fig_p020_20.png]
Figure 22
Figure 22. Figure 22: Sample difficulty score distributions on [PITH_FULL_IMAGE:figures/full_fig_p021_22.png]
Figure 21
Figure 21. Figure 21: Sample difficulty score distributions on SST [PITH_FULL_IMAGE:figures/full_fig_p021_21.png]
Figure 23
Figure 23. Figure 23: Sample difficulty score distributions on [PITH_FULL_IMAGE:figures/full_fig_p021_23.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

68 extracted references · 32 canonical work pages

  1. [1]

    Bo An. 2023. https://doi.org/10.1145/3603168 Prompt-based for low-resource tibetan text classification . ACM Trans. Asian Low-Resour. Lang. Inf. Process., 22(8)

    work page doi:10.1145/3603168 2023

  2. [2]

    Abdul Hameed Azeemi, Ihsan Ayyub Qazi, and Agha Ali Raza. 2025. https://aclanthology.org/2025.coling-main.206/ To label or not to label: Hybrid active learning for neural machine translation . In Proceedings of the 31st International Conference on Computational Linguistics, pages 3071--3082, Abu Dhabi, UAE. Association for Computational Linguistics

    work page 2025

  3. [3]

    Guangji Bai, Zheng Chai, Chen Ling, Shiyu Wang, Jiaying Lu, Nan Zhang, Tingwei Shi, Ziyang Yu, Mengdan Zhu, Yifei Zhang, Xinyuan Song, Carl Yang, Yue Cheng, and Liang Zhao. 2024. https://arxiv.org/abs/2401.00625 Beyond efficiency: A systematic survey of resource-efficient large language models . Preprint, arXiv:2401.00625

    arXiv 2024

  4. [4]

    Yoshua Bengio, J \' e r \^ o me Louradour, Ronan Collobert, and Jason Weston. 2009. https://doi.org/10.1145/1553374.1553380 Curriculum learning . In Proceedings of the 26th Annual International Conference on Machine Learning, ICML 2009, Montreal, Quebec, Canada, June 14-18, 2009 , volume 382 of ACM International Conference Proceeding Series , pages 41--48. ACM

    arXiv 2009

  5. [5]

    Yu, Qiang Yang, and Xing Xie

    Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, Wei Ye, Yue Zhang, Yi Chang, Philip S. Yu, Qiang Yang, and Xing Xie. 2024. https://doi.org/10.1145/3641289 A survey on evaluation of large language models . ACM Transactions on Intelligent Systems and Technology, 15(3)

    doi:10.1145/3641289 2024

  6. [6]

    Jiaao Chen, Dinghan Shen, Weizhu Chen, and Diyi Yang. 2021. https://doi.org/10.18653/v1/2021.acl-long.338 H idden C ut: Simple data augmentation for natural language understanding with better generalizability . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural ...

    work page doi:10.18653/v1/2021.acl-long.338 2021

  7. [7]

    Bowman, Holger Schwenk, and Veselin Stoyanov

    Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel R. Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. Xnli: Evaluating cross-lingual sentence representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics

    2018

  8. [8]

    Cunningham, Dominique Archambault, and Austin Kung

    Sophia R. Cunningham, Dominique Archambault, and Austin Kung. 2024. https://doi.org/10.36227/techrxiv.171651876.65094225/v1 Efficient training and inference: Techniques for large language models using llama

    work page arXiv 2024

Show all 68 references
  1. [9]

    Kunal Dahiya, Nilesh Gupta, Deepak Saini, Akshay Soni, Yajun Wang, Kushal Dave, Jian Jiao, Gururaj K, Prasenjit Dey, Amit Singh, and 1 others. 2023. Ngame: Negative mining-aware mini-batching for extreme classification. In Proceedings of the Sixteenth ACM International Confere...

    2023

  2. [10]

    Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In Proceedings of the 11th International AAAI Conference on Web and Social Media, ICWSM '17, pages 512--515

    2017

  3. [11]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

    2018 arXiv

  4. [12]

    Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi - Min Chan, Weize Chen, Jing Yi, Weilin Zhao, Xiaozhi Wang, Zhiyuan Liu, Hai - Tao Zheng, Jianfei Chen, Yang Liu, Jie Tang, Juanzi Li, and Maosong Sun. 2023. https://doi.org/1...

    2023 doi

  5. [13]

    Bonaventure F. P. Dossou, Atnafu Lambebo Tonja, Oreen Yousuf, Salomey Osei, Abigail Oppong, Iyanuoluwa Shode, Oluwabusayo Olufunke Awoyomi, and Chris Emezue. 2022. https://doi.org/10.18653/v1/2022.sustainlp-1.11 A fro LM : A self-active learning-based multilingual pretrained l...

    2022 doi

  6. [14]

    Jingfei Du, Edouard Grave, Beliz Gunel, Vishrav Chaudhary, Onur Celebi, Michael Auli, Veselin Stoyanov, and Alexis Conneau. 2021. https://doi.org/10.18653/v1/2021.naacl-main.426 Self-training improves pre-training for natural language understanding . In Proceedings of the 2021...

    2021 doi

  7. [15]

    Shen Gao, Zhengliang Shi, Minghang Zhu, Bowen Fang, Xin Xin, Pengjie Ren, Zhumin Chen, Jun Ma, and Zhaochun Ren. 2024. https://doi.org/10.1609/AAAI.V38I16.29759 Confucius: Iterative tool learning from introspection feedback by easy-to-difficult curriculum . In Thirty-Eighth AA...

    2024 doi

  8. [16]

    Tianyu Gao, Adam Fisch, and Danqi Chen. 2021. https://doi.org/10.18653/v1/2021.acl-long.295 Making pre-trained language models better few-shot learners . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint...

    2021 doi

  9. [17]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022 a . https://openreview.net/forum?id=nZeVKeeFYf9 Lora: Low-rank adaptation of large language models . In The Tenth International Conference on Learning Represen...

    2022

  10. [18]

    Linmei Hu, Zeyi Liu, Ziwang Zhao, Lei Hou, Liqiang Nie, and Juanzi Li. 2024. https://doi.org/10.1109/TKDE.2023.3310002 A survey of knowledge enhanced pre-trained language models . IEEE Transactions on Knowledge and Data Engineering, 36(4):1413--1430

    2024

  11. [19]

    Shengding Hu, Ning Ding, Huadong Wang, Zhiyuan Liu, Jingang Wang, Juanzi Li, Wei Wu, and Maosong Sun. 2022 b . https://doi.org/10.18653/v1/2022.acl-long.158 Knowledgeable prompt-tuning: Incorporating knowledge into prompt verbalizer for text classification . In Proceedings of ...

    2022 doi

  12. [20]

    Jianheng Huang, Leyang Cui, Ante Wang, Chengyi Yang, Xinting Liao, Linfeng Song, Junfeng Yao, and Jinsong Su. 2024. https://doi.org/10.18653/v1/2024.acl-long.77 Mitigating catastrophic forgetting in large language models with self-synthesized rehearsal . In Proceedings of the ...

    2024 doi

  13. [21]

    Borna Jafarpour, Dawn Sepehr, and Nick Pogrebnyakov. 2021. https://doi.org/10.18653/v1/2021.internlp-1.6 Active curriculum learning . In Proceedings of the First Workshop on Interactive Learning for Natural Language Processing, pages 40--45, Online. Association for Computation...

    2021 doi

  14. [22]

    Lu Jiang, Deyu Meng, Qian Zhao, Shiguang Shan, and Alexander Hauptmann. 2015. Self-paced curriculum learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 29

    2015

  15. [23]

    Woojeong Jin, Yu Cheng, Yelong Shen, Weizhu Chen, and Xiang Ren. 2022. https://doi.org/10.18653/v1/2022.acl-long.197 A good prompt is worth millions of parameters: Low-resource prompt-based learning for vision-language models . In Proceedings of the 60th Annual Meeting of the ...

    2022 doi

  16. [24]

    Zixuan Ke, Bing Liu, Nianzu Ma, Hu Xu, and Lei Shu. 2021. https://proceedings.neurips.cc/paper/2021/hash/bcd0049c35799cdf57d06eaf2eb3cff6-Abstract.html Achieving forgetting prevention and knowledge transfer in continual learning . In Advances in Neural Information Processing S...

    2021

  17. [25]

    Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. 2017. https://doi.org/10.1073/pnas.1...

    2017 doi

  18. [26]

    Teven Le Scao and Alexander Rush. 2021. https://doi.org/10.18653/v1/2021.naacl-main.208 How many data points is a prompt worth? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pa...

    2021 doi

  19. [27]

    Lewis and William A

    David D. Lewis and William A. Gale. 1994. https://arxiv.org/abs/cmp-lg/9407020 A sequential algorithm for training text classifiers . CoRR, abs/cmp-lg/9407020

    1994 arXiv

  20. [28]

    Jia Li, Chongyang Tao, Wei Wu, Yansong Feng, Dongyan Zhao, and Rui Yan. 2019. https://doi.org/10.18653/v1/D19-1128 Sampling matters! an empirical study of negative sampling strategies for learning of matching models in retrieval-based dialogue systems . In Proceedings of the 2...

    2019 doi

  21. [29]

    Yihong Liu, Haotian Ye, Chunlan Ma, Mingyang Wang, and Hinrich Schütze. 2024. https://arxiv.org/abs/2409.18199 Langsamp: Language-script aware multilingual pretraining . Preprint, arXiv:2409.18199

    2024 arXiv

  22. [30]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. https://arxiv.org/abs/1907.11692 Roberta: A robustly optimized bert pretraining approach . Preprint, arXiv:1907.11692

    2019 arXiv

  23. [31]

    Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101

    2017 arXiv

  24. [32]

    Bolei Ma, Ercong Nie, Helmut Schmid, and Hinrich Schuetze. 2023. https://aclanthology.org/2023.konvens-main.1/ Is prompt-based finetuning always better than vanilla finetuning? insights from cross-lingual language understanding . In Proceedings of the 19th Conference on Natura...

    2023

  25. [33]

    Adyasha Maharana and Mohit Bansal. 2022. https://doi.org/10.18653/v1/2022.naacl-main.72 On curriculum learning for commonsense reasoning . In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techn...

    2022 doi

  26. [34]

    Gomez, Adrien Morisot, Sebastian Farquhar, and Yarin Gal

    Sören Mindermann, Jan Brauner, Muhammed Razzak, Mrinank Sharma, Andreas Kirsch, Winnie Xu, Benedikt Höltgen, Aidan N. Gomez, Adrien Morisot, Sebastian Farquhar, and Yarin Gal. 2022. https://arxiv.org/abs/2206.07137 Prioritized training on points that are learnable, worth learn...

    2022 arXiv

  27. [35]

    Moin Nadeem, Tianxing He, Kyunghyun Cho, and James Glass. 2020. https://doi.org/10.18653/v1/2020.aacl-main.36 A systematic characterization of sampling algorithms for open-ended language generation . In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Assoc...

    2020 doi

  28. [36]

    Marwa Na \"i r, Kamel Yamani, Lynda Lhadj, and Riyadh Baghdadi. 2024. https://doi.org/10.18653/v1/2024.acl-srw.44 Curriculum learning for small code language models . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student ...

    2024 doi

  29. [37]

    Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Naveed Akhtar, Nick Barnes, and Ajmal Mian. 2024. https://arxiv.org/abs/2307.06435 A comprehensive overview of large language models . Preprint, arXiv:2307.06435

    2024 arXiv

  30. [38]

    Jerzy Neyman. 1934. http://www.jstor.org/stable/2342192 On the two different aspects of the representative method: The method of stratified sampling and the method of purposive selection . Journal of the Royal Statistical Society, 97(4):558--625

    1934

  31. [39]

    Emmanouil Antonios Platanios, Otilia Stretcu, Graham Neubig, Barnabas Poczos, and Tom Mitchell. 2019. https://doi.org/10.18653/v1/N19-1119 Competence-based curriculum learning for neural machine translation . In Proceedings of the 2019 Conference of the North A merican Chapter...

    2019 doi

  32. [40]

    Longhua Qian, Guodong Zhou, Fang Kong, and Qiaoming Zhu. 2009. https://aclanthology.org/D09-1149/ Semi-supervised learning for semantic relation classification using stratified sampling strategy . In Proceedings of the 2009 Conference on Empirical Methods in Natural Language P...

    2009

  33. [41]

    Leonardo Ranaldi, Giulia Pucci, and Fabio Massimo Zanzotto. 2023. https://aclanthology.org/2023.ranlp-1.101/ Modeling easiness for training transformers with curriculum learning . In Proceedings of the 14th International Conference on Recent Advances in Natural Language Proces...

    2023

  34. [42]

    Timo Schick and Hinrich Sch \"u tze. 2021 a . https://doi.org/10.18653/v1/2021.eacl-main.20 Exploiting cloze-questions for few-shot text classification and natural language inference . In Proceedings of the 16th Conference of the European Chapter of the Association for Computa...

    2021 doi

  35. [43]

    Timo Schick and Hinrich Sch \"u tze. 2021 b . https://doi.org/10.18653/v1/2021.emnlp-main.32 Few-shot text generation with natural language instructions . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 390--402, Online and Punt...

    2021 doi

  36. [44]

    Timo Schick and Hinrich Sch \"u tze. 2021 c . https://doi.org/10.18653/v1/2021.naacl-main.185 It`s not just size that matters: Small language models are also few-shot learners . In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computat...

    2021 doi

  37. [45]

    Yunfan Shao, Linyang Li, Zhaoye Fei, Hang Yan, Dahua Lin, and Xipeng Qiu. 2024. https://doi.org/10.18653/v1/2024.findings-acl.833 Balanced data sampling for language model training with clustering . In Findings of the Association for Computational Linguistics: ACL 2024, pages ...

    2024 doi

  38. [46]

    Logan IV, Eric Wallace, and Sameer Singh

    Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.346 A uto P rompt: E liciting K nowledge from L anguage M odels with A utomatically G enerated P rompts . In Proceedings of the 2020 Conference o...

    2020 doi

  39. [47]

    Manning, Andrew Ng, and Christopher Potts

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. https://aclanthology.org/D13-1170/ Recursive deep models for semantic compositionality over a sentiment treebank . In Proceedings of the 2013 Conference on Em...

    2013

  40. [48]

    Linxin Song, Jieyu Zhang, Tianxiang Yang, and Masayuki Goto. 2022. https://doi.org/10.18653/v1/2022.findings-emnlp.119 Adaptive ranking-based sample selection for weakly supervised class-imbalanced text classification . In Findings of the Association for Computational Linguist...

    2022 doi

  41. [49]

    Petru Soviany, Radu Tudor Ionescu, Paolo Rota, and Nicu Sebe. 2022. https://arxiv.org/abs/2101.10382 Curriculum learning: A survey . Preprint, arXiv:2101.10382

    2022 arXiv

  42. [50]

    Yamshchikov

    Maxim Surkov, Vladislav Mosin, and Ivan P. Yamshchikov. 2022. https://doi.org/10.18653/v1/2022.insights-1.16 Do data-based curricula work? In Proceedings of the Third Workshop on Insights from Negative Results in NLP, pages 119--128, Dublin, Ireland. Association for Computatio...

    2022 doi

  43. [51]

    Qingyu Tan, Lu Xu, Lidong Bing, and Hwee Tou Ng. 2023. https://doi.org/10.18653/v1/2023.findings-acl.549 Class-adaptive self-training for relation extraction with incompletely annotated training data . In Findings of the Association for Computational Linguistics: ACL 2023, pag...

    2023 doi

  44. [52]

    Megh Thakkar, Tolga Bolukbasi, Sriram Ganapathy, Shikhar Vashishth, Sarath Chandar, and Partha Talukdar. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.125 Self-influence guided data reweighting for language model pre-training . In Proceedings of the 2023 Conference on Empi...

    2023 doi

  45. [53]

    Faizad Ullah, Ubaid Azam, Ali Faheem, Faisal Kamiran, and Asim Karim. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.449 Comparing prompt-based and standard fine-tuning for U rdu text classification . In Findings of the Association for Computational Linguistics: EMNLP 2...

    2023 doi

  46. [54]

    Han Wang, Canwen Xu, and Julian McAuley. 2022. https://doi.org/10.18653/v1/2022.naacl-main.401 Automatic multi-label prompting: Simple and interpretable few-shot classification . In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computa...

    2022 doi

  47. [55]

    Hao Wang, Minghua Nuo, and Shan Jiang. 2025. https://aclanthology.org/2025.coling-main.38/ Knowledge graph entity typing with curriculum contrastive learning . In Proceedings of the 31st International Conference on Computational Linguistics, pages 574--583, Abu Dhabi, UAE. Ass...

    2025

  48. [56]

    Xiaoxia Wu, Ethan Dyer, and Behnam Neyshabur. 2021. https://openreview.net/forum?id=tW4QEInpni When do curricula work? In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . OpenReview.net

    2021

  49. [57]

    Zhipeng Xie and Yahe Li. 2024. https://aclanthology.org/2024.lrec-main.445/ Discriminative language model as semantic consistency scorer for prompt-based few-shot text classification . In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Lang...

    2024

  50. [58]

    Benfeng Xu, Licheng Zhang, Zhendong Mao, Quan Wang, Hongtao Xie, and Yongdong Zhang. 2020. https://doi.org/10.18653/v1/2020.acl-main.542 Curriculum learning for natural language understanding . In Proceedings of the 58th Annual Meeting of the Association for Computational Ling...

    2020 doi

  51. [59]

    Zhichao Yang, Shufan Wang, Bhanu Pratap Singh Rawat, Avijit Mitra, and Hong Yu. 2022. https://doi.org/10.18653/v1/2022.findings-emnlp.127 Knowledge injected prompt based fine-tuning for multi-label few-shot ICD coding . In Findings of the Association for Computational Linguist...

    2022 doi

  52. [60]

    Hongbin Ye, Ningyu Zhang, Shumin Deng, Xiang Chen, Hui Chen, Feiyu Xiong, Xi Chen, and Huajun Chen. 2022. https://doi.org/10.1145/3485447.3511921 Ontology-enhanced prompt-tuning for few-shot learning . In Proceedings of the ACM Web Conference 2022, WWW ’22. ACM

    2022

  53. [61]

    Donghan Yu, Chenguang Zhu, Yiming Yang, and Michael Zeng. 2022 a . https://doi.org/10.1609/AAAI.V36I10.21417 JAKET: joint pre-training of knowledge graph and language understanding . In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conferenc...

    2022 doi

  54. [62]

    Yue Yu, Lingkai Kong, Jieyu Zhang, Rongzhi Zhang, and Chao Zhang. 2022 b . Actune: Uncertainty-based active self-training for active fine-tuning of pretrained language models. In Proceedings of the 2022 conference of the North American chapter of the association for computatio...

    2022

  55. [63]

    Michelle Yuan, Hsuan-Tien Lin, and Jordan Boyd-Graber. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.637 Cold-start active learning through self-supervised language modeling . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)...

    2020 doi

  56. [64]

    Zhisong Zhang, Emma Strubell, and Eduard Hovy. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.414 A survey of active learning for natural language processing . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6166--6190, Abu D...

    2022 doi

  57. [65]

    Qingyan Zhao, Ruifang He, Jinpeng Zhang, Chang Liu, and Bo Wang. 2024 a . https://aclanthology.org/2024.lrec-main.1217/ Representation degeneration problem in prompt-based models for natural language understanding . In Proceedings of the 2024 Joint International Conference on ...

    2024

  58. [66]

    Raoyuan Zhao, Abdullatif K \"o ksal, Yihong Liu, Leonie Weissweiler, Anna Korhonen, and Hinrich Schuetze. 2024 b . https://doi.org/10.18653/v1/2024.findings-emnlp.412 S ynth E val: Hybrid behavioral testing of NLP models with synthetic C heck L ists . In Findings of the Associ...

    2024 doi

  59. [67]

    Qingqing Zhu, Xiuying Chen, Pengfei Wu, JunFei Liu, and Dongyan Zhao. 2021. https://doi.org/10.18653/v1/2021.findings-emnlp.111 Combining curriculum learning and knowledge distillation for dialogue generation . In Findings of the Association for Computational Linguistics: EMNL...

    2021 doi

  60. [68]

    Yutao Zhu, Jian - Yun Nie, Yixuan Su, Haonan Chen, Xinyu Zhang, and Zhicheng Dou. 2022. https://doi.org/10.1145/3511808.3557328 From easy to hard: A dual curriculum learning framework for context-aware document ranking . In Proceedings of the 31st ACM International Conference ...

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.