REVIEW 2 cited by
SuperLoRA: Parameter-Efficient Unified Adaptation of Multi-Layer Attention Modules
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Low-rank adaptation (LoRA) and its variants are widely employed in fine-tuning large models, including large language models for natural language processing and diffusion models for computer vision. This paper proposes a generalized framework called SuperLoRA that unifies and extends different LoRA variants, which can be realized under different hyper-parameter settings. Introducing grouping, folding, shuffling, projecting, and tensor factoring, SuperLoRA offers high flexibility compared with other LoRA variants and demonstrates superior performance for transfer learning tasks especially in the extremely few-parameter regimes.
Forward citations
Cited by 2 Pith papers
-
Regularizing Subspace Redundancy of Low-Rank Adaptation
ReSoRA adds a penalty that reduces redundancy among rank-1 subspaces of LoRA-style adapters, producing modest accuracy improvements on vision-language retrieval and visual classification.
-
KARST: Multi-Kernel Kronecker Adaptation with Re-Scaling Transmission for Visual Classification
A sum of low-rank Kronecker adapters plus channel-wise re-scaling gives small average accuracy gains over prior PEFT methods on visual classification benchmarks.
Discussion (0). Continue with ORCID to comment.