Pith. sign in

REVIEW 3 cited by

Data Augmentation using Pre-trained Transformer Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2003.02245 v2 pith:F5NIT36W submitted 2020-03-04 cs.CL cs.LG

Varun Kumar , Ashutosh Choudhary , Eunah Cho This is my paper · ORCID
classification cs.CLcs.LG
keywords modelsdatapre-trainedaugmentationdifferentmodelbertmethods
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Language model based pre-trained models such as BERT have provided significant gains across different NLP tasks. In this paper, we study different types of transformer based pre-trained models such as auto-regressive models (GPT-2), auto-encoder models (BERT), and seq2seq models (BART) for conditional data augmentation. We show that prepending the class labels to text sequences provides a simple yet effective way to condition the pre-trained models for data augmentation. Additionally, on three classification benchmarks, pre-trained Seq2Seq model outperforms other data augmentation methods in a low-resource setting. Further, we explore how different pre-trained model based data augmentation differs in-terms of data diversity, and how well such methods preserve the class-label information.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Backtranslation and paraphrasing in the LLM era? Comparing data augmentation methods for emotion classification

    cs.CL 2025-07 conditional novelty 6.0 of 10

    Backtranslation and paraphrasing produce competitive or better classification gains than zero-shot and few-shot generation when augmenting a low-resource emotion dataset.

  2. The Synthetic Mirror -- Synthetic Data at the Age of Agentic AI

    cs.CY 2025-06 conditional novelty 4.0 of 10

    A position paper claiming existing data and AI regulations are unprepared for synthetic data generated by and for agentic AI, recommending targeted legal amendments and new standards.

  3. Explainable AI: XAI-Guided Context-Aware Data Augmentation

    cs.CL 2025-06 conditional novelty 4.0 of 10

    XAI-guided augmentation that replaces the least important words, identified by Integrated Gradients, with back-translated synonyms or paraphrases improves hate speech and sentiment classification accuracy by up to 8 p...

Pith tools