REVIEW 8 cited by
AdapterHub: A Framework for Adapting Transformers
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The current modus operandi in NLP involves downloading and fine-tuning pre-trained models consisting of millions or billions of parameters. Storing and sharing such large trained models is expensive, slow, and time-consuming, which impedes progress towards more general and versatile NLP methods that learn from and for many tasks. Adapters -- small learnt bottleneck layers inserted within each layer of a pre-trained model -- ameliorate this issue by avoiding full fine-tuning of the entire model. However, sharing and integrating adapter layers is not straightforward. We propose AdapterHub, a framework that allows dynamic "stitching-in" of pre-trained adapters for different tasks and languages. The framework, built on top of the popular HuggingFace Transformers library, enables extremely easy and quick adaptations of state-of-the-art pre-trained models (e.g., BERT, RoBERTa, XLM-R) across tasks and languages. Downloading, sharing, and training adapters is as seamless as possible using minimal changes to the training scripts and a specialized infrastructure. Our framework enables scalable and easy access to sharing of task-specific models, particularly in low-resource scenarios. AdapterHub includes all recent adapter architectures and can be found at https://AdapterHub.ml.
Forward citations
Cited by 8 Pith papers
-
Test-Time Scaling via Error Localization
TTEL uses feedback-induced token probability drops to localize the first error in a failed reasoning trace and branch a new generation from that prefix, improving pass@k per token on coding and math benchmarks.
-
One Prompt, Many Sounds: Modeling Listener Variability in LLM-Based Equalization
LLMs using in-context learning and fine-tuning on listener experiment data generate equalization settings that align better with population preferences than random sampling or static presets.
-
Video-Guided Text-to-Music Generation Using Public Domain Movie Collections
OSSL is the first self-hosted, mood-annotated video-music dataset, and a video adapter on MusicGen-Medium improves film music generation over text-only baselines.
-
DA-VPT: Semantic-Guided Visual Prompt Tuning for Vision Transformers
DA-VPT guides visual prompts with a proxy-anchor metric loss and dynamic class-to-prompt clustering, reporting consistent gains over VPT baselines across classification and segmentation.
-
Exploring Adapter Design Tradeoffs for Low Resource Music Generation
Adapter placement, architecture, and size strongly change generation quality and cost for MusicGen and Mustango on two non-Western genres, with late-layer, mid-sized (40M) adapters reported as the best tradeoff.
-
PrunePEFT: Iterative Hybrid Pruning for Parameter-Efficient Fine-tuning of LLMs
An iterative hybrid pruning method selects which PEFT modules to keep at each transformer layer, matching or improving fixed PEFT baselines on GLUE at 1% trainable parameters.
-
Schema as Parameterized Tools for Universal Information Extraction
SPT turns predefined IE schemas into trainable token embeddings, enabling one small model to handle closed, open, and on-demand extraction through schema retrieval, generation, and infilling.
-
GEM: A Scale-Aware and Distribution-Sensitive Sparse Fine-Tuning Framework for Effective Downstream Adaptation
GEM selects fine-tuning parameters by gradient-to-weight ratio and distributes the budget by layer entropy, reaching 0.1% parameter updates with small accuracy gains on several NLP tasks.
Discussion (0). Continue with ORCID to comment.