REVIEW 33 cited by
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Prompt tuning, which only tunes continuous prompts with a frozen language model, substantially reduces per-task storage and memory usage at training. However, in the context of NLU, prior work reveals that prompt tuning does not perform well for normal-sized pretrained models. We also find that existing methods of prompt tuning cannot handle hard sequence labeling tasks, indicating a lack of universality. We present a novel empirical finding that properly optimized prompt tuning can be universally effective across a wide range of model scales and NLU tasks. It matches the performance of finetuning while having only 0.1%-3% tuned parameters. Our method P-Tuning v2 is an implementation of Deep Prompt Tuning \cite{li2021prefix,qin2021learning} optimized and adapted for NLU. Given the universality and simplicity of P-Tuning v2, we believe it can serve as an alternative to finetuning and a strong baseline for future research.Our code and data are released at https://github.com/THUDM/P-tuning-v2.
Forward citations
Cited by 33 Pith papers
-
Dual Information Speech Language Models for Emotional Conversations
A dual-adapter design with equivalence replacement regularization lets frozen LLMs perceive both paralinguistic and linguistic information from speech for emotional conversation.
-
GrocLM: Grocery Category Recommendation in E-Commerce with Large Language Models
A LoRA-tuned LLM with trie-constrained decoding improves grocery category recommendation and yields a 7.5% cart-add lift in production.
-
Meta-cavity Quantum Electrodynamics
Quantum dots in ultrathin geometric-phase metacavities emit triggered single photons that are simultaneously Purcell-enhanced and wavefront-shaped into vortices, holograms, or spin-locked beams.
-
MHA-RAG: Improving Efficiency, Accuracy, and Consistency by Encoding Exemplars as Soft Prompts
MHA-RAG encodes retrieved exemplars into order-invariant soft prompts via multi-head attention, claiming ~20-point effective-accuracy gains over RAG at ~10x lower inference FLOPs.
-
Latent Activation Editing: Inference-Time Refinement of Learned Policies for Safer Multirobot Navigation
Editing a frozen RL policy's latent activations at inference time, using a collision world model, cuts collisions by about 90% on a curated set of hard multirotor scenarios and on real Crazyflies.
-
Towards Meta-Cognitive Knowledge Editing for Multimodal LLMs
CogEdit and MIND shift multimodal knowledge editing toward evaluating and enabling meta-cognitive skills: self-awareness, boundary monitoring, and noise robustness.
-
Adversarial Topic-aware Prompt-tuning for Cross-topic Automated Essay Scoring
ATOP uses shared and topic-specific soft prompts with adversarial training and pseudo-labels to improve cross-topic automated essay scoring, reporting better QWK scores than nine baselines on ASAP++.
-
Adversarial Demonstration Learning for Low-resource NER Using Dual Similarity
ADELL improves few-shot NER by combining dual similarity (semantic plus learned feature Jaccard) for demonstration selection with adversarial demonstration training that forces the tagger to attend to demonstrations.
-
Animation Needs Attention: A Holistic Approach to Slides Animation Comprehension with Visual-Language Models
A vision-language model fine-tuned on a new synthetic slide-animation dataset outperforms GPT-4.1 and Gemini-2.5-Pro at describing slide animations, especially on synthetic evaluation data.
-
Graft: Integrating the Domain Knowledge via Efficient Parameter Synergy for MLLMs
Graft merges two domain-specialized multimodal models by combining channel-wise gating, entropy-based global weighting, and an activation compatibility score to improve fusion without retraining.
-
Test3R: Learning to Reconstruct 3D at Test Time
Test3R improves 3D reconstruction by optimizing visual prompts at test time so that pointmaps from different image pairs are geometrically consistent.
-
Shaking to Reveal: Perturbation-Based Detection of LLM Hallucinations
SSP adds a learned, sample-specific noise prompt to an LLM input and scores hallucination by the cosine shift in intermediate representations, outperforming output-confidence baselines on QA benchmarks.
-
Assortment of Attention Heads: Accelerating Federated PEFT with Head Pruning and Strategic Client Selection
A federated fine-tuning method prunes 90% of attention heads, weights updates by attention importance, and selects clients by loss gap, cutting communication 1.8x and training compute 3.9x with under 2% accuracy drop.
-
DA-VPT: Semantic-Guided Visual Prompt Tuning for Vision Transformers
DA-VPT guides visual prompts with a proxy-anchor metric loss and dynamic class-to-prompt clustering, reporting consistent gains over VPT baselines across classification and segmentation.
-
Privacy-preserving Prompt Personalization in Federated Learning for Multimodal Large Language Models
SecFPP combines hierarchical prompt decomposition with secret-sharing-based adaptive clustering to protect user prompts in federated learning while preserving personalization accuracy.
-
PMA: Towards Parameter-Efficient Point Cloud Understanding via Point Mamba Adapter
PMA adapts frozen point cloud models by ordering and fusing all intermediate layer features with Mamba, achieving parameter-efficient gains on ScanObjectNN, ModelNet40, and ShapeNetPart.
-
Can Multimodal Large Language Models Understand Spatial Relations?
SpatialMQA, a new spatial-relation benchmark, shows the top MLLM reaches 48.14% accuracy versus 98.40% for humans.
-
Data-Efficient Adaptation of LLMs via Attention Head Reweighting
Learning a single scalar per attention head lets LLMs adapt to few-shot text classification better than LoRA, with 200–1000x fewer trainable parameters.
-
SOS-LoRA: Static Orthogonal-Subspace Low-Rank Adaptation with Fixed Multi-Scale Scaling
A LoRA update split into several fixed, differently-scaled low-rank experts with orthogonal input directions improves fine-tuning accuracy at the same parameter count.
-
AI-driven Remote Facial Skin Hydration and TEWL Assessment from Selfie Images: A Systematic Solution
Selfie images can be used to predict skin hydration and water loss with R2 up to about 0.35, using a new dataset of 336 panelists and an adapter-based vision transformer.
-
CLAImate: AI-Enabled Climate Change Communication through Personalized and Localized Narrative Visualizations
A personalized, localized AI conversation system for climate communication shows modest factual accuracy and positive early feedback from 10 UK users.
-
Heterogeneous Graph Prompt Learning via Adaptive Weight Pruning
GPAWP prunes low-importance graph prompts using gradient-based importance masks and retunes the remaining prompts, improving few-shot node classification over HGPrompt on ACM, DBLP, and Freebase.
-
EdgeLoRA: An Efficient Multi-Tenant LLM Serving System on Edge Devices
EdgeLoRA combines automatic adapter routing, LRU caching with a memory pool, and grouped LoRA batching to serve thousands of LoRA adapters on edge devices with up to 4x higher throughput than llama.cpp.
-
CLONE: Customizing LLMs for Efficient Latency-Aware Inference at the Edge
CLONE combines generative pruning, LoRA adapters, a parameter-free Mixture-of-Experts router, and learning-based DVFS to make LLM inference on edge devices faster and more energy-efficient, claiming up to 11.92x speed...
-
FLoE: Fisher-Based Layer Selection for Efficient Sparse Adaptation of Low-Rank Experts
FLoE uses Fisher information to pick the transformer layers that matter and a Bayesian optimizer to set LoRA rank, cutting trainable parameters while keeping or improving accuracy.
-
Quaff: Quantized Parameter-Efficient Fine-Tuning under Outlier Spatial Stability Hypothesis
Quaff shows that activation outlier channels keep their spatial positions during LLM fine-tuning, and exploits this stability to cut fine-tuning memory and latency with INT8 quantization while matching or beating full...
-
Zero-Residual Concept Erasure via Progressive Alignment in Text-to-Image Model
A closed-form concept erasure method that enforces zero alignment residual in the optimization objective and applies updates progressively across layers to better preserve generation quality.
-
Foundation Models and Transformers for Anomaly Detection: A Survey
A taxonomy and literature review of Transformer-based visual anomaly detection, compromised by fabricated citations with dummy arXiv IDs.
-
SI-Agent: An Agentic Framework for Feedback-Driven Generation and Tuning of Human-Readable System Instructions for Large Language Models
The paper proposes a multi-agent loop (instructor, follower, feedback) to auto-generate human-readable system prompts, claiming good benchmark performance and readability, but the supporting experiments are not reprod...
-
Beyond the Sentence: A Survey on Context-Aware Machine Translation with Large Language Models
A survey of context-aware machine translation with large language models, categorizing prompting, fine-tuning, and agent-based approaches.
-
OSoRA: Output-Dimension and Singular-Value Initialized Low-Rank Adaptation
OSoRA fine-tunes LLMs by updating only singular values and one output-dimension vector, using frozen singular vectors from an SVD of the pretrained weights.
-
Progtuning: Progressive Fine-tuning Framework for Transformer-based Language Models
A progressive scheduling trick that updates only the last remaining blocks in later epochs reduces parameter-update counts by about 25% with roughly unchanged GLUE and SQuAD scores.
-
PL-FGSA: A Prompt Learning Framework for Fine-Grained Sentiment Analysis Based on MindSpore
PL-FGSA claims a prompt-learning TextCNN framework for fine-grained sentiment analysis, but the reported results lack baseline comparisons and few-shot validation.
Discussion (0). Sign in to comment.