Pith. sign in

REVIEW 5 cited by

Revisiting Pre-Trained Models for Chinese Natural Language Processing

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2004.13922 v2 pith:S77DQWE3 submitted 2020-04-29 cs.CL

Yiming Cui , Wanxiang Che , Ting Liu , Bing Qin , Shijin Wang , Guoping Hu This is my paper
classification cs.CL
keywords languagepre-trainedchinesemacbertmodelstasksmodelproposed
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Bidirectional Encoder Representations from Transformers (BERT) has shown marvelous improvements across various NLP tasks, and consecutive variants have been proposed to further improve the performance of the pre-trained language models. In this paper, we target on revisiting Chinese pre-trained language models to examine their effectiveness in a non-English language and release the Chinese pre-trained language model series to the community. We also propose a simple but effective model called MacBERT, which improves upon RoBERTa in several ways, especially the masking strategy that adopts MLM as correction (Mac). We carried out extensive experiments on eight Chinese NLP tasks to revisit the existing pre-trained language models as well as the proposed MacBERT. Experimental results show that MacBERT could achieve state-of-the-art performances on many NLP tasks, and we also ablate details with several findings that may help future research. Resources available: https://github.com/ymcui/MacBERT

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. BlossomPsy: A User-Centric AI System for Adaptive and Engaging MBTI Personality Assessments

    cs.HC 2026-07 conditional novelty 5.0 of 10

    BlossomPsy combines multi-turn LLM dialogue, photo-based questions, a multi-head classifier, and a modified UCB bandit algorithm to deliver MBTI assessments with higher user engagement and preliminary consistency with...

  2. MExplore: an entity-based visual analytics approach for medical expertise acquisition

    cs.HC 2025-07 conditional novelty 5.0 of 10

    An entity-based multi-level visual analytics system for acquiring medical expertise from unstructured medical text, evaluated in a small user study without significance testing.

  3. MEDTalk: Multimodal Controlled 3D Facial Animation with Dynamic Emotions by Disentangled Embedding

    cs.CV 2025-07 conditional novelty 5.0 of 10

    A 3D facial animation framework that disentangles content and emotion and predicts frame-wise emotion intensity from audio plus text for dynamic expressions.

  4. Overview of the NLPCC 2025 Shared Task: Gender Bias Mitigation Challenge

    cs.CL 2025-06 conditional novelty 4.0 of 10

    A Chinese gender-bias corpus and shared-task benchmark show detection and classification are feasible, while automatic mitigation remains weak.

  5. LLM Encoder vs. Decoder: Robust Detection of Chinese AI-Generated Text with LoRA

    cs.CL 2025-08 conditional novelty 3.0 of 10

    On the NLPCC 2025 Chinese AI-text detection benchmark, LoRA-adapted Qwen2.5-7B reaches 95.94% test accuracy, beating BERT-large (79.3%), RoBERTa-large (76.3%), and FastText (83.5%).

Pith tools