Controlled ablations of 38 models find MLM superior to CLM on representation benchmarks while CLM offers better data efficiency and stability; a biphasic CLM-then-MLM schedule is optimal under fixed compute and improves when initialized from pretrained CLM models.
Revisiting few-sample bert fine-tuning
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
representative citing papers
GRAIN is a gradient aggregation method using min-norm objectives to ensure non-negative inner products with group gradients, yielding tighter uniform stability bounds than SGD under smoothness assumptions.
RAG-adapted LLaMA-3-8B outperforms both baseline and fine-tuned models on expert-rated accuracy (75.5%), relevance (90.8%), and overall preference (85.2%) for additive manufacturing questions.
citing papers explorer
-
Should We Still Pretrain Encoders with Masked Language Modeling?
Controlled ablations of 38 models find MLM superior to CLM on representation benchmarks while CLM offers better data efficiency and stability; a biphasic CLM-then-MLM schedule is optimal under fixed compute and improves when initialized from pretrained CLM models.
-
GRAIN: Group Aggregation via Min-Norm Objective
GRAIN is a gradient aggregation method using min-norm objectives to ensure non-negative inner products with group gradients, yielding tighter uniform stability bounds than SGD under smoothness assumptions.
-
Domain Adaptation of Large Language Models for Polymer-Composite Additive Manufacturing Using Retrieval-Augmented Generation and Fine-Tuning
RAG-adapted LLaMA-3-8B outperforms both baseline and fine-tuned models on expert-rated accuracy (75.5%), relevance (90.8%), and overall preference (85.2%) for additive manufacturing questions.