REVIEW 1 cited by
Continual Training of Language Models for Few-Shot Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Recent work on applying large language models (LMs) achieves impressive performance in many NLP applications. Adapting or posttraining an LM using an unlabeled domain corpus can produce even better performance for end-tasks in the domain. This paper proposes the problem of continually extending an LM by incrementally post-train the LM with a sequence of unlabeled domain corpora to expand its knowledge without forgetting its previous skills. The goal is to improve the few-shot end-task learning in these domains. The resulting system is called CPT (Continual PostTraining), which to our knowledge, is the first continual post-training system. Experimental results verify its effectiveness.
Forward citations
Cited by 1 Pith paper
-
AggTruth: Contextual Hallucination Detection using Aggregated Attention Scores in LLMs
AggTruth uses four ways of aggregating attention scores over the retrieved passage to train a logistic regression detector, achieving stable cross-task AUROC across four LLMs.
Discussion (0). Continue with ORCID to comment.