Pith. sign in

REVIEW 7 cited by

Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRA

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2304.06027 v2 pith:XIENFPMM submitted 2023-04-12 cs.CV cs.AIcs.LG

James Seale Smith , Yen-Chang Hsu , Lingyu Zhang , Ting Hua , Zsolt Kira , Yilin Shen , Hongxia Jin This is my paper · ORCID
classification cs.CVcs.AIcs.LG
keywords continualdiffusionc-loracustomizationtext-to-imageconceptsmodelsonly
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Recent works demonstrate a remarkable ability to customize text-to-image diffusion models while only providing a few example images. What happens if you try to customize such models using multiple, fine-grained concepts in a sequential (i.e., continual) manner? In our work, we show that recent state-of-the-art customization of text-to-image models suffer from catastrophic forgetting when new concepts arrive sequentially. Specifically, when adding a new concept, the ability to generate high quality images of past, similar concepts degrade. To circumvent this forgetting, we propose a new method, C-LoRA, composed of a continually self-regularized low-rank adaptation in cross attention layers of the popular Stable Diffusion model. Furthermore, we use customization prompts which do not include the word of the customized object (i.e., "person" for a human face dataset) and are initialized as completely random embeddings. Importantly, our method induces only marginal additional parameter costs and requires no storage of user data for replay. We show that C-LoRA not only outperforms several baselines for our proposed setting of text-to-image continual customization, which we refer to as Continual Diffusion, but that we achieve a new state-of-the-art in the well-established rehearsal-free continual learning setting for image classification. The high achieving performance of C-LoRA in two separate domains positions it as a compelling solution for a wide range of applications, and we believe it has significant potential for practical impact. Project page: https://jamessealesmith.github.io/continual-diffusion/

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. HyperDCM: Dynamic Cluster Memory Replay in Hyperbolic Space for Continual Robotic Navigation Across Scenes

    cs.RO 2026-07 reject novelty 6.0 of 10

    A scene-graph-plus-hyperbolic replay memory reduces reported performance drops in continual diffusion navigation, but the headline Drop metric is defined across different scenes and does not clearly measure forgetting.

  2. PROL : Rehearsal Free Continual Learning in Streaming Data via Prompt Online Learning

    cs.LG 2025-07 conditional novelty 6.0 of 10

    PROL achieves state-of-the-art rehearsal-free online continual learning accuracy on CIFAR100, ImageNet-R, ImageNet-A, and CUB with a single lightweight prompt generator and 16 trainable numbers per class.

  3. LoRP-TTS: Low-Rank Personalized Text-To-Speech

    cs.SD 2025-02 conditional novelty 6.0 of 10

    LoRP-TTS shows that per-prompt LoRA fine-tuning with one short recording improves speaker similarity in Voicebox-based zero-shot TTS, at some cost in inference time.

  4. Skill Expansion and Composition in Parameter Space

    cs.LG 2025-02 conditional novelty 6.0 of 10

    PSEC shows that weighting and summing LoRA skill modules inside a diffusion policy network outperforms composing the same skills in action or noise space across D4RL, DSRL, DMC, and Meta-World tasks.

  5. CKAA: Cross-subspace Knowledge Alignment and Aggregation for Robust Continual Learning

    cs.CV 2025-07 conditional novelty 5.0 of 10

    CKAA aligns features and decision boundaries across task-specific subspaces and mixes adapters by task confidence, improving class-incremental learning accuracy under misidentified task labels.

  6. SplitLoRA: Balancing Stability and Plasticity in Continual Learning Through Gradient Space Splitting

    cs.LG 2025-05 reject novelty 5.0 of 10

    SplitLoRA picks the LoRA update subspace size from previous-task gradient singular values using a hyperparameter alpha, and freezes the projection to keep updates in that subspace.

  7. Continual Learning for Generative AI: From LLMs to MLLMs and Beyond

    cs.LG 2025-06 conditional novelty 4.0 of 10

    A survey that categorizes continual learning methods for generative models into architecture-based, regularization-based, and replay-based paradigms across four model families.

Pith tools