Pith. sign in

REVIEW 12 cited by

One-for-All: Generalized LoRA for Parameter-Efficient Fine-tuning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2306.07967 v2 pith:5ITIMXHN submitted 2023-06-13 cs.LG cs.AIcs.CV

Arnav Chavan , Zhuang Liu , Deepak Gupta , Eric Xing , Zhiqiang Shen This is my paper
classification cs.LGcs.AIcs.CV
keywords gloralorageneralizedtasksactivationsadaptationdomainfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We present Generalized LoRA (GLoRA), an advanced approach for universal parameter-efficient fine-tuning tasks. Enhancing Low-Rank Adaptation (LoRA), GLoRA employs a generalized prompt module to optimize pre-trained model weights and adjust intermediate activations, providing more flexibility and capability across diverse tasks and datasets. Moreover, GLoRA facilitates efficient parameter adaptation by employing a scalable, modular, layer-wise structure search that learns individual adapter of each layer. Originating from a unified mathematical formulation, GLoRA exhibits strong transfer learning, few-shot learning and domain generalization abilities, as it adapts to new tasks through not only weights but also additional dimensions like activations. Comprehensive experiments demonstrate that GLoRA outperforms all previous methods in natural, specialized, and structured vision benchmarks, achieving superior accuracy with fewer parameters and computations. The proposed method on LLaMA-1 and LLaMA-2 also show considerable enhancements compared to the original LoRA in the language domain. Furthermore, our structural re-parameterization design ensures that GLoRA incurs no extra inference cost, rendering it a practical solution for resource-limited applications. Code and models are available at: https://github.com/Arnav0400/ViT-Slim/tree/master/GLoRA.

Discussion (0). Sign in to comment.

Forward citations

Cited by 12 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. REAL-OW: Rehearsal-free Open World Object Detection with Low-Rank Adaptation and Dual-Stage Objectness Modeling

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A rehearsal-free open-world detector using collaborative LoRA adapters and dual-stage objectness modeling outperforms exemplar-replay OWOD methods on standard benchmarks.

  2. TARA: Token-Aware LoRA for Composable Personalization in Diffusion Models

    cs.CV 2025-08 conditional novelty 6.0 of 10

    TARA adds token-focused masking and a token alignment loss to LoRA adapters, allowing several independently trained personalized adapters to be composed with less identity loss and feature leakage.

  3. ConsNoTrainLoRA: Data-driven Weight Initialization of Low-rank Adapters using Constraints

    cs.CV 2025-07 conditional novelty 6.0 of 10

    LoRA adapters can be initialized with a closed-form estimate derived from constraint sets linking source and target activations, improving fine-tuning speed and accuracy.

  4. Beyond Low-Rank Tuning: Model Prior-Guided Rank Allocation for Effective Transfer in Low-Data and Large-Gap Regimes

    cs.CV 2025-06 conditional novelty 6.0 of 10

    SR-LoRA sets each LoRA layer's rank to the stable rank of that layer's pretrained weight matrix, improving few-shot transfer on large domain gaps without rank search.

  5. DA-VPT: Semantic-Guided Visual Prompt Tuning for Vision Transformers

    cs.CV 2025-05 conditional novelty 6.0 of 10

    DA-VPT guides visual prompts with a proxy-anchor metric loss and dynamic class-to-prompt clustering, reporting consistent gains over VPT baselines across classification and segmentation.

  6. MultLFG: Training-free Multi-LoRA composition using Frequency-domain Guidance

    cs.CV 2025-05 conditional novelty 6.0 of 10

    MultLFG merges multiple LoRA adapters by adaptively weighting them in wavelet frequency subbands per denoising timestep, improving multi-concept composition on the ComposLoRA benchmark compared to prior training-free methods.

  7. A Language-Guided Bayesian Optimization for Efficient LoRA Hyperparameter Search

    cs.CL 2026-01 conditional novelty 5.0 of 10

    LLM embeddings plus Bayesian optimization find better LoRA hyperparameters in ~30 proxy trials than standard published settings.

  8. Language-Aware Information Maximization for Transductive Few-Shot CLIP

    cs.CV 2025-08 conditional novelty 5.0 of 10

    LIMO, a transductive loss combining mutual information, zero-shot KL regularization, and LoRA, sets new state-of-the-art few-shot accuracy for CLIP on 11 datasets.

  9. EdgeLoRA: An Efficient Multi-Tenant LLM Serving System on Edge Devices

    cs.DC 2025-07 conditional novelty 5.0 of 10

    EdgeLoRA combines automatic adapter routing, LRU caching with a memory pool, and grouped LoRA batching to serve thousands of LoRA adapters on edge devices with up to 4x higher throughput than llama.cpp.

  10. Dynamic Context-oriented Decomposition for Task-aware Low-rank Adaptation with Less Forgetting and Faster Convergence

    cs.LG 2025-06 conditional novelty 5.0 of 10

    CorDA++ uses data-driven SVD to initialize LoRA adapters, adding per-layer covariance selection and rank allocation that reduce forgetting and speed convergence compared to LoRA, PiSSA, QLoRA, and other baselines.

  11. Parameter-Efficient Fine-Tuning of 3D DDPM for MRI Image Generation Using Tensor Networks

    eess.IV 2025-07 conditional novelty 4.0 of 10

    TenVOO represents 3D convolution weight updates as tensor networks, fine-tuning a brain MRI DDPM with only 0.3% of full trainable parameters while achieving competitive or better structural similarity on ADNI, PPMI, a...

  12. Regularized Low-Rank Adaptation for Few-Shot Organ Segmentation

    cs.CV 2025-07 reject novelty 4.0 of 10

    ARENA, a low-rank adaptation method with l1-regularized singular values, reportedly improves few-shot organ segmentation, but its described update rule is internally inconsistent.

Pith tools