Pith. sign in

REVIEW 3 cited by

CLEAR: Contrastive Learning for Sentence Representation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2012.15466 v1 pith:VV6O4IEM submitted 2020-12-31 cs.CL

Zhuofeng Wu , Sinong Wang , Jiatao Gu , Madian Khabsa , Fei Sun , Hao Ma This is my paper
classification cs.CL
keywords sentencecontrastivelearningrepresentationaugmentationscleardifferentlanguage
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Pre-trained language models have proven their unique powers in capturing implicit language features. However, most pre-training approaches focus on the word-level training objective, while sentence-level objectives are rarely studied. In this paper, we propose Contrastive LEArning for sentence Representation (CLEAR), which employs multiple sentence-level augmentation strategies in order to learn a noise-invariant sentence representation. These augmentations include word and span deletion, reordering, and substitution. Furthermore, we investigate the key reasons that make contrastive learning effective through numerous experiments. We observe that different sentence augmentations during pre-training lead to different performance improvements on various downstream tasks. Our approach is shown to outperform multiple existing methods on both SentEval and GLUE benchmarks.

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Adapting Pretrained Language Models for Citation Classification via Self-Supervised Contrastive Learning

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Citss combines sentence-level cropping and keyphrase perturbation with contrastive learning to fine-tune both encoder and decoder language models for citation classification.

  2. SemCSE: Semantic Contrastive Sentence Embeddings Using LLM-Generated Summaries For Scientific Abstracts

    cs.CL 2025-07 conditional novelty 5.0 of 10

    Using multiple LLM-generated summaries of the same abstract as positive pairs trains scientific text embeddings that beat citation-trained baselines on retrieval and clustering, while the new benchmark shares its trai...

  3. Learning Text Styles: A Study on Transfer, Attribution, and Verification

    cs.CL 2025-07 conditional novelty 3.0 of 10

    A thesis compiles published work claiming that lightweight adapters, contrastive disentanglement, and instruction tuning improve text style transfer, authorship attribution, and authorship verification.

Pith tools