REVIEW 2 cited by
Quantum-PEFT: Ultra parameter-efficient fine-tuning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
This paper introduces Quantum-PEFT that leverages quantum computations for parameter-efficient fine-tuning (PEFT). Unlike other additive PEFT methods, such as low-rank adaptation (LoRA), Quantum-PEFT exploits an underlying full-rank yet surprisingly parameter efficient quantum unitary parameterization. With the use of Pauli parameterization, the number of trainable parameters grows only logarithmically with the ambient dimension, as opposed to linearly as in LoRA-based PEFT methods. Quantum-PEFT achieves vanishingly smaller number of trainable parameters than the lowest-rank LoRA as dimensions grow, enhancing parameter efficiency while maintaining a competitive performance. We apply Quantum-PEFT to several transfer learning benchmarks in language and vision, demonstrating significant advantages in parameter efficiency.
Forward citations
Cited by 2 Pith papers
-
RefLoRA: Refactored Low-Rank Adaptation for Efficient Fine-Tuning of Large Models
RefLoRA picks a per-step optimal low-rank factorization (a matrix geometric mean) that balances LoRA's factors, improving fine-tuning convergence and accuracy.
-
$\mu$-MoE: Test-Time Pruning as Micro-Grained Mixture-of-Experts
Test-time Wanda pruning, reframed as a mixture of micro-experts, adapts the sparse weight mask to each prompt and improves perplexity and VQA accuracy over static pruning baselines.
Discussion (0). Continue with ORCID to comment.