SSVD: Structured SVD for Parameter-Efficient Fine-Tuning and Benchmarking under Domain Shift in ASR
Reviewed by Pithpith:GI22ES2Aopen to challenge →
read the original abstract
Parameter-efficient fine-tuning (PEFT) has emerged as a scalable solution for adapting large foundation models. While low-rank adaptation (LoRA) is widely used in speech applications, its state-of-the-art variants, e.g., VeRA, DoRA, PiSSA, and SVFT, are developed mainly for language and vision tasks, with limited validation in speech. This work presents the first comprehensive integration and benchmarking of these PEFT methods within ESPnet. We further introduce structured SVD-guided (SSVD) fine-tuning, which selectively rotates input-associated right singular vectors while keeping output-associated vectors fixed to preserve semantic mappings. This design enables robust domain adaptation with minimal trainable parameters and improved efficiency. We evaluate all methods on domain-shifted speech recognition tasks, including child speech and dialectal variation, across model scales from 0.1B to 2B. All implementations are released in ESPnet to support reproducibility and future work.
This paper has not been read by Pith yet.
Forward citations
Cited by 2 Pith papers
-
GC-LoRA: Gated Convolutional LoRA for Parameter-Efficient Acoustic Adaptation
GC-LoRA introduces a gated convolutional adapter into LoRA for efficient adaptation of transformer speech models to domain-specific acoustics, reporting up to 10.9% WER reduction on diverse test sets.
-
Parameter-Efficient Continual Learning for Automatic Speech Recognition
A PECL method for ASR partitions weight matrices via singular values, adapts only via rotations in the tail subspace, and averages rotations across tasks to reduce forgetting while outperforming baselines on two benchmarks.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.