Pith. sign in

REVIEW 12 cited by

GPT Understands, Too

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2103.10385 v2 pith:I4DGHULB submitted 2021-03-18 cs.CL cs.LG

Xiao Liu , Yanan Zheng , Zhengxiao Du , Ming Ding , Yujie Qian , Zhilin Yang , Jie Tang This is my paper
classification cs.CLcs.LG
keywords languagediscretep-tuningperformancepromptseffectivenaturalprompt
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Prompting a pretrained language model with natural language patterns has been proved effective for natural language understanding (NLU). However, our preliminary study reveals that manual discrete prompts often lead to unstable performance -- e.g., changing a single word in the prompt might result in substantial performance drop. We propose a novel method P-Tuning that employs trainable continuous prompt embeddings in concatenation with discrete prompts. Empirically, P-Tuning not only stabilizes training by minimizing the gap between various discrete prompts, but also improves performance by a sizeable margin on a wide range of NLU tasks including LAMA and SuperGLUE. P-Tuning is generally effective for both frozen and tuned language models, under both the fully-supervised and few-shot settings.

Discussion (0). Sign in to comment.

Forward citations

Cited by 12 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Vision Transformer Finetuning Benefits from Non-Smooth Components

    cs.LG 2026-02 conditional novelty 6.0 of 10

    For vision transformers, components with higher input-output sensitivity (attention and feedforward layers) yield better and more stable fine-tuning accuracy than smoother LayerNorm components.

  2. Modeling Code: Is Text All You Need?

    cs.AI 2025-07 conditional novelty 6.0 of 10

    A GNN-encoded LLVM IR graph, prepended as soft prompts to a frozen code LLM, improves accuracy on device mapping, algorithm classification, vulnerability detection, and code translation tasks.

  3. Adversarial Demonstration Learning for Low-resource NER Using Dual Similarity

    cs.CL 2025-07 conditional novelty 6.0 of 10

    ADELL improves few-shot NER by combining dual similarity (semantic plus learned feature Jaccard) for demonstration selection with adversarial demonstration training that forces the tagger to attend to demonstrations.

  4. Time Series Foundation Models for Multivariate Financial Time Series Forecasting

    q-fin.GN 2025-07 reject novelty 6.0 of 10

    Pretrained TTM shows large transfer and sample-efficiency gains in three financial forecasting tasks relative to training from scratch, but methodological flaws including possible look-ahead bias weaken the quantitati...

  5. Impact of Fine-Tuning Methods on Memorization in Large Language Models

    cs.CL 2025-06 conditional novelty 6.0 of 10

    Prompt-based fine-tuning (prefix, prompt, P-tuning) shows consistently lower membership inference AUC than parameter-based fine-tuning (full, head, LoRA) across GPT-2, LLaMA-2-7B, and LLaMA-3-1B.

  6. MOPSA: Mixture of Prompt-Experts Based Speaker Adaptation for Elderly Speech Recognition

    eess.AS 2025-05 conditional novelty 5.0 of 10

    MOPSA uses K-means clustered speaker prompts with a trained router to provide zero-shot, real-time Whisper adaptation for elderly speech, achieving WER/CER reductions on DementiaBank Pitt and JCCOCC MoCA.

  7. Speech as a Multimodal Digital Phenotype for Multi-Task LLM-based Mental Health Prediction

    cs.CL 2025-05 conditional novelty 5.0 of 10

    A trimodal, longitudinal, multi-task LLM pipeline predicts adolescent depression with 70.8% balanced accuracy on the private DEW dataset, but the gain over simpler baselines is modest and lacks external validation.

  8. TokenVerse++: Towards Flexible Multitask Learning with Dynamic Task Activation

    cs.CL 2025-08 conditional novelty 4.0 of 10

    Adding task-specific learned vectors to acoustic embeddings lets a transducer ASR model train on partially labeled data, matching or beating the fully labeled TokenVerse baseline on most tasks.

  9. Breaking Physical and Linguistic Borders: Multilingual Federated Prompt Tuning for Low-Resource Languages

    cs.CL 2025-07 conditional novelty 4.0 of 10

    Federated averaging of prompt embeddings from a frozen multilingual model improves accuracy on some low-resource tasks (XNLI) but not consistently on others (MasakhaNEWS).

  10. Leveraging Self-Attention for Input-Dependent Soft Prompting in LLMs

    cs.CL 2025-06 conditional novelty 4.0 of 10

    ID-SPAM generates input-dependent soft prompts with a self-attention mechanism and a two-layer MLP, and shows modest gains over several soft-prompt baselines on NLU tasks.

  11. CoLA: Collaborative Low-Rank Adaptation

    cs.CL 2025-05 conditional novelty 4.0 of 10

    CoLA generalizes LoRA to multiple A and B matrices with a principal-component initialization and reports gains of roughly 2-4 accuracy points over PiSSA on low-sample fine-tuning benchmarks.

  12. Optimising Language Models for Downstream Tasks: A Post-Training Perspective

    cs.CL 2025-06 conditional novelty 3.0 of 10

    A dissertation that repackages the author's previously published papers on continued pre-training, prompt tuning, and instruction modelling into a single narrative.

Pith tools