Pith. sign in

REVIEW 4 major objections 5 minor 72 references

LoRA-Loop: Closing the Synthetic Replay Cycle for Continual VLM Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A feedback loop—LoRA-adapting the frozen diffusion generator and filtering replay by the updated VLM's confidence—closes the domain and semantic gap in synthetic replay, improving continual vision-language learning on MTIL.

desk verdict Sensible incremental extension of GIFT, but the SOTA claim rests on sub-noise margins without error bars. read the letter →

arxiv 2507.13568 v2 pith:QJOGUICQ submitted 2025-07-17 cs.CV

Kaihong Wang , Donghyun Kim , Margrit Betke This is my paper
classification cs.CV
keywords continuallearningvision-languagemodelssyntheticreplaylow-rankadaptationStableDiffusionzero-shotgeneralizationtask-incrementalconfidence-basedfiltering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Continual learning of vision-language models faces a tension: finetuning on new tasks erases old knowledge, and synthetic replay—generating past-class images with Stable Diffusion—has been a storage-light remedy, but the generator often misses the fine-grained, domain-specific details of real task data. This paper argues that the gap can be closed by making the generator itself task-aware. After each finetuning step, the updated VLM's confidence scores are used twice: to pick the real examples that train a task-specific LoRA adapter on a frozen Stable Diffusion model, and then to filter the generated candidates that go into the distillation replay set. On the Multi-domain Task Incremental Learning (MTIL) benchmark, the resulting LoRA-Loop pipeline improves over the prior GIFT baseline on both task orders across transfer, average, and final accuracy, while preserving zero-shot generalization. The authors' claim is that this feedback loop, not extra real data, is what lets replay distill knowledge faithfully without vaulting storage or privacy costs.

What carries the argument

The machinery is the feedback loop itself, implemented as two confidence-ranked selections around one low-rank adaptation. LoRA (low-rank adaptation) is a parameter-efficient update that adds two small matrices $A\in\mathbb{R}^{d\times r}$, $B\in\mathbb{R}^{r\times d}$ to a frozen weight matrix so that $W'=W_0+AB$ with rank $r\ll d$; here it adapts the text-to-image generator while leaving Stable Diffusion's base weights and its broad priors untouched. At task $i$, the updated VLM $f^{i-1}$ scores every candidate image by cosine similarity between the image embedding and the prompt embedding, keeping the top-$k$ per class for the replay set; then the freshly finetuned VLM $f^i$ scores the real training examples, and for each class the $l$ most prototypical and $l$ most borderline examples train the task-specific LoRA adapter $A_i$ used for all future sampling of that class's concepts. The loop therefore uses the VLM's own uncertainty as the alignment signal, and the claim is that this turns a generic generator into a task-specialized one for the classes that need replay.

What would settle it

Train the LoRA adapter and filter replay candidates using a fidelity score that is independent of the VLM—for example, a separately trained classifier or human ratings on a 1000-image sample—while keeping everything else identical; if the gains over GIFT vanish or invert under the independent scorer but the VLM-scored variant keeps them, the reported benefit is driven by self-confirmation rather than by closing the domain and semantic gap.

Watch

Extended reading notes

Core claim

The central claim is that synthetic replay for continual VLM learning fails when the generator and the task distribution drift apart, and that this drift can be corrected by a closed loop: the finetuned VLM reports where it is confident, and that signal steers both the adaptation of the generator and the choice of replay samples. Concretely, the paper injects a rank-4 LoRA adapter into a frozen Stable Diffusion v1.5, trains it on the per-class prototypical and boundary real examples selected by post-finetuning CLIP confidence, generates eight candidates per class, keeps the single highest-confidence image per class for distillation, and feeds the GIFT distillation losses. On MTIL Order I this yields 69.8% transfer, 77.6% average, and 86.0% final accuracy, edging out GIFT by 0.1, 0.3, and 0.6 percentage points respectively, with comparable margins on Order II. The paper further reports that both modules—LoRA finetuning and confidence filtering—contribute independently, that a two-example-per-class adapter matches the replay quality of storing two real images per class at roughly a quarter of the storage, and that freezing the base generator while adapting only the low-rank matrices protects the VLM's zero-shot behavior.

Load-bearing premise

The load-bearing premise is that the VLM's cosine-similarity confidence score is a trustworthy measure of whether a generated image truly matches the task's class semantics and domain, so that selecting and training on the highest-confidence samples improves replay fidelity rather than reinforcing the model's existing biases.

Editorial extensions

If this is right

  • Swapping the LoRA-adapted generator into existing replay pipelines such as GIFT is sufficient to raise final-task accuracy and average performance, without changing the distillation losses or the training schedule.
  • Task-specific adapters can be accumulated per class set, so later tasks can replay earlier classes by selecting the matching stored adapter rather than regenerating from the base model.
  • The two-stage selection isolates two failure modes: filtering removes grossly misaligned generations, while LoRA finetuning closes subtler domain and semantic gaps that filtering alone cannot see.
  • Synthetic replay at two generated samples per class nearly matches two stored real images per class on distillation quality, but at 30.79 MB versus 118.95 MB of storage when adapters replace real buffers.
  • Because the base generator is frozen and only low-rank adapters are trained, the approach keeps the VLM's zero-shot transfer essentially flat while improving average and last-task accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable check on the mechanism is whether the confidence filter selects for genuine task alignment or for the VLM's prior agreement: score the replay with an independent oracle, such as human ratings or a held-out classifier, and see whether the reported gains persist.
  • The same loop should transfer to other text-to-image generators and to other VLM backbones, because it only requires a frozen generator, a scorer, and a low-rank adapter; a natural next step is to vary the scorer's calibration and measure how the replay quality moves with it.
  • Because the reported gains over GIFT are small (0.3–0.6 points on average and last accuracy), the sharper test of the claim is on tasks with larger domain or semantic shifts than MTIL, where the generator's misalignment is more severe and the loop should give a bigger margin.
  • The paper's real-replay comparison suggests a practical policy: keep a few real examples per class only when privacy allows, and otherwise rely on LoRA-adapted synthetic replay, which nearly matches it while avoiding linear storage growth; combining a very small real buffer for calibration with synthetic replay for mass could be a further improvement.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes LoRA-Loop, a continual-learning method for vision-language models that augments the GIFT synthetic-replay pipeline with task-specific LoRA adapters trained on a small set of real examples, plus a two-stage confidence-based selection mechanism. The generator is a frozen Stable Diffusion model; after each task, a LoRA adapter is finetuned on high- and low-confidence real examples selected by the updated VLM, and future replay candidates generated for previous classes are filtered by the current VLM's image-text cosine similarity before distillation. The method is evaluated on the MTIL benchmark under two task orders, with reported improvements over GIFT and other baselines on Transfer, Avg., and Last metrics, along with ablations, hyperparameter sensitivity studies, and qualitative comparisons on Aircraft and DTD.

Significance. If the reported gains are statistically robust, the paper makes a useful and conceptually clean contribution: it closes the loop from VLM finetuning back to the generator, requiring only a small LoRA adapter per task and no storage of real rehearsal images. The drop-in integration with GIFT is attractive, and the ablation design, hyperparameter sweep, and qualitative evidence are appropriate for a systems-style continual-learning paper. The main weakness is that the central empirical claim of state-of-the-art performance rests on single-run comparisons whose margins (0.1 to 0.6 percentage points) are within the typical seed-to-seed variability of CLIP finetuning on this benchmark. The self-referential confidence criterion is also not validated against an external measure of sample fidelity. With multi-seed evaluation and a more careful treatment of the confidence proxy, the work could be a solid practical contribution.

major comments (4)
  1. [Tables 1 and 2, Sec. 4.2.1] The state-of-the-art claim is not statistically grounded. The reported margins over the reproduced GIFT baseline are +0.1/+0.3/+0.6 pp (Transfer/Avg/Last) on Order I and +0.2/+0.1/+0.3 pp on Order II, but no standard deviations, number of seeds, confidence intervals, or significance tests are reported anywhere. In CLIP finetuning on benchmarks of this size, seed variation is typically several tenths of a point, so an n=1 comparison at these margins cannot support the abstract's claim that LoRA-Loop 'outperforms' prior synthetic-replay techniques. Please report mean and standard deviation over at least three to five independent runs and, if possible, a paired significance test or confidence intervals for the differences against GIFT.
  2. [Algorithm 1, Steps 1 and 3; Secs. 3.2.1-3.2.2] The confidence-based selection uses the very VLM that is being continually finetuned: Step 1 filters generated replay candidates by cosine similarity computed with the previous VLM f^{i-1}, and Step 3 selects LoRA training examples using the updated VLM f^i. This is a self-referential loop: the model is selecting the data that will be used to train itself, and high CLIP confidence may reflect the model's own biases rather than true alignment with task data. In particular, for replay filtering, the criterion may preferentially select synthetic images that the current model already classifies well, which would reduce the replay benefit. To support the central claim that confidence tracks sample fidelity, please provide a direct validation, for example by measuring the correlation between the CLIP confidence score and an external alignment judgment (e.g., human ratings or a held-out classifier's correctness) on a sample of generated images, or by comparing against an oracle-based selection criterion on a subset of the benchmark.
  3. [Table 3, Sec. 4.2.2] The ablation table has misaligned component checkmarks: the rows are formatted as '+DST +AWC +LFT +SF', but the checkmark columns do not line up with these headers, so it is impossible to determine which configuration each row represents. This directly undermines the text's component-isolation claims, such as 'LoRA finetuning is particularly effective at preserving previously learned knowledge' and 'sample filtering more consistently maintains zero-shot generalization.' Please reformat the table so each row's active components are unambiguous, and additionally note that the isolated component deltas (0.1 to 0.4 pp) are again single-run numbers that need the same multi-seed treatment as the main tables.
  4. [Table 5, Sec. 4.3.2] The comparison to real replay is presented as showing that LoRA-Loop 'nearly matches' real-data replay, but the table actually reports that 2 real replay examples per class beat LoRA-Loop on Avg (77.6 vs. 77.0) and Last (86.9 vs. 85.9) with no overlap analysis or error bars. The conclusion that real buffering harms generalization is based on Transfer differences of -0.1 to +0.1 pp, which are not statistically meaningful in a single run. Please either qualify these conclusions appropriately or provide repeated-run results for this comparison.
minor comments (5)
  1. [Table 7, Appendix] In the Order II detailed Transfer row for LoRA-Loop, the entries for DTD and EuroSAT are both reported as 69.4 and the CIFAR100 entry appears to be missing, which makes the row internally inconsistent and prevents verification of the reported average of 66.3. Please correct the table.
  2. [Algorithm 1 and Sec. 3.2.1] The notation for the VLM used in confidence scoring is inconsistent: Step 1 refers to the 'frozen VLM from the last round f^{i-1}' while Step 3 uses f^i; please clarify that these are the then-current checkpoints and state explicitly which parameters are frozen at each scoring stage.
  3. [Sec. 4.1, Implementation Details] The generation procedure uses Stable Diffusion v1.5 with a guidance scale of 7.5 and 50 denoising steps, but no random seed or sampling mechanism is reported; please specify whether the Mpre candidates are generated with fixed seeds and across how many runs, since this affects reproducibility.
  4. [References] References [67] and [68] cite the same paper with identical titles and authors; please merge or distinguish them correctly.
  5. [Sec. 4.2.2, Table 4] The hyperparameter sweep in Table 4f reports that Mpre=16 does not improve over Mpre=8, and Table 4a shows a steady drop for r>4; these statements are supported by the table, but a sentence clarifying whether the plateau at Mpre=16 is within expected noise would be helpful given the single-run setting.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the confidence-based selection is a sampling heuristic, and the central claim is evaluated on held-out real test sets rather than on the selection score itself.

full rationale

LoRA-Loop's derivation chain is: (1) select real examples by VLM confidence; (2) LoRA-finetune Stable Diffusion on them; (3) generate candidates and filter them by VLM confidence; (4) distill the VLM with GIFT losses on the filtered set; (5) report MTIL Transfer/Avg/Last on held-out real test sets. No step defines its output as its input or fits a parameter that is then renamed as a prediction. The filter score conf_j = cos(f_img(x), f_txt(p)) is used as a selection heuristic; 'fidelity' and 'alignment' are not defined as that score, and the paper supports improved replay quality with qualitative comparisons against real task images and downstream accuracy gains, not with the filter score itself. The framework builds on GIFT [56] and ZSCL [68], both external works, and there are no load-bearing self-citations or imported uniqueness theorems. The reported comparisons lack error bars and hyperparameters are tuned on MTIL Order I, which are genuine validity concerns (test-set tuning and seed variance), but they are not circularity: the final numbers are not algebraically forced by a fitted quantity or by a definitional identity. The only self-referential aspect is that the same VLM scores candidates and is later distilled on them; if that score were miscalibrated, replay could become self-confirming, but that is a robustness hypothesis rather than an exhibited reduction of the paper's claim to its inputs. Therefore no circular step is identified.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The method's reported gains depend on several hand-chosen hyperparameters and on two unvalidated assumptions: that VLM confidence measures generation quality, and that two examples per class can adapt Stable Diffusion. No new entities are introduced, and no code or data are provided to pin down the exact recipe.

free parameters (6)
  • LoRA rank r = 4
    Chosen by sweep in Table 4a; rank above 4 degrades all metrics, so the reported gain depends on this choice.
  • LoRA training set size l = 2 per class
    Selected via Table 4b; at r=16, larger l does not help. The method's ability to adapt the generator from two examples is a core premise.
  • Replay sampling budget Mpre = 8 candidates per class
    Swept in Table 4f; performance plateaus at 8, so the cost and benefit claims are tied to this budget.
  • Top-k replay selection k = 1 per class
    Fixed to one retained sample per class after confidence ranking; controls memory but also caps the diversity of the replay set.
  • LoRA finetuning epochs and learning rate = 100 epochs, learning rate 1e-4
    Reported in implementation details; no sensitivity analysis is given for these, but they affect whether the adapted generator remains stable.
  • Selection policies (Top & Bottom for LoRA data, Top for replay) = ablated choices
    Tables 4d and 4e show these policies outperform alternatives; they are choices tuned on the benchmark rather than derived from first principles.
assumptions (4)
  • ad hoc to paper CLIP cosine similarity between a generated image and its prompt is a valid proxy for sample fidelity and task alignment.
    Used in Algorithm 1 Steps 1 and 3 to filter both real and synthetic samples; never validated against human judgment or generation quality metrics, and it creates a self-referential loop with the finetuned VLM.
  • domain assumption A rank-4 LoRA trained on two real examples per class can steer Stable Diffusion to capture fine-grained semantic and domain patterns.
    Central to the method; supported only by qualitative figures (Figures 2 and 3), not by quantitative generation fidelity measures.
  • domain assumption The MTIL benchmark with two task orders is representative of real-world continual VLM learning.
    Results are shown only on MTIL; generalization to other benchmarks or deployment settings is assumed without evidence.
  • domain assumption GIFT's distillation and AWC losses remain sound when the replay generator is swapped for a LoRA-adapted one.
    The framework builds directly on GIFT without re-deriving these losses; the integration is asserted in Section 3.2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LoRA-Loop: Closing the Synthetic Replay Cycle for Continual VLM Learning." pith.science (2026). https://pith.science/paper/QJOGUICQ

@misc{pith2026250713568,
  author       = {Pith},
  title        = {Pith review of: LoRA-Loop: Closing the Synthetic Replay Cycle for Continual VLM Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QJOGUICQ}},
  note         = {Machine review of arXiv:2507.13568}
}
read the original abstract

Continual learning for vision-language models has achieved remarkable performance through synthetic replay, where samples are generated using Stable Diffusion to regularize during finetuning and retain knowledge. However, real-world downstream applications often exhibit domain-specific nuances and fine-grained semantics not captured by generators, causing synthetic-replay methods to produce misaligned samples that misguide finetuning and undermine retention of prior knowledge. In this work, we propose a LoRA-enhanced synthetic-replay framework that injects task-specific low-rank adapters into a frozen Stable Diffusion model, efficiently capturing each new task's unique visual and semantic patterns. Specifically, we introduce a two-stage, confidence-based sample selection: we first rank real task data by post-finetuning VLM confidence to focus LoRA finetuning on the most representative examples, then generate synthetic samples and again select them by confidence for distillation. Our approach integrates seamlessly with existing replay pipelines-simply swap in the adapted generator to boost replay fidelity. Extensive experiments on the Multi-domain Task Incremental Learning (MTIL) benchmark show that our method outperforms previous synthetic-replay techniques, achieving an optimal balance among plasticity, stability, and zero-shot capability. These results demonstrate the effectiveness of generator adaptation via LoRA for robust continual learning in VLMs.

Figures

Figures reproduced from arXiv: 2507.13568 by the authors.

Figure 1
Figure 1. Comparison of baseline synthetic replay (top) versus our [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Visualization of the LoRA finetuning data and generation samples on Aircraft. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Visualization of the LoRA finetuning data and generation samples on DTD. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

72 extracted references · 60 canonical work pages

  1. [1]

    Expert gate: Lifelong learning with a network of experts

    Rahaf Aljundi, Punarjay Chakravarty, and Tinne Tuytelaars. Expert gate: Lifelong learning with a network of experts. In CVPR, pages 7120–7129, 2017. 2

    work page 2017

  2. [2]

    Memory aware synapses: Learning what (not) to forget

    Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In ECCV, 2018. 2

    work page 2018

  3. [3]

    Food-101 - mining discriminative components with random forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101 - mining discriminative components with random forests. In ECCV, 2014. 4

    work page 2014

  4. [4]

    Sharegpt4v: Improving large multi-modal models with better captions

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-modal models with better captions. In ECCV, 2024. 3

    work page 2024

  5. [5]

    Describing textures in the wild

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In CVPR, pages 3606–3613, 2014. 4

    work page 2014

  6. [6]

    The MNIST database of handwritten digit images for machine learning research [best of the web].IEEE Signal Processing Magazine, 29, 2012

    Li Deng. The MNIST database of handwritten digit images for machine learning research [best of the web].IEEE Signal Processing Magazine, 29, 2012. 4

    work page 2012

  7. [7]

    Learning without mem- orizing

    Prithviraj Dhar, Rajat Vikram Singh, Kuan-Chuan Peng, Ziyan Wu, and Rama Chellappa. Learning without mem- orizing. In CVPR, pages 5138–5146, 2019. 2

    2019

  8. [8]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021. 4

    work page 2021

Show all 72 references
  1. [9]

    Podnet: Pooled outputs distil- lation for small-tasks incremental learning

    Arthur Douillard, Matthieu Cord, Charles Ollion, Thomas Robert, and Eduardo Valle. Podnet: Pooled outputs distil- lation for small-tasks incremental learning. In ECCV, 2020. 2

    2020

  2. [10]

    Dytox: Transformers for continual learn- ing with dynamic token expansion

    Arthur Douillard, Alexandre Ram ´e, Guillaume Couairon, and Matthieu Cord. Dytox: Transformers for continual learn- ing with dynamic token expansion. In CVPR, pages 9275– 9285, 2022. 2

    2022

  3. [11]

    Scaling laws of synthetic images for model training

    Lijie Fan, Kaifeng Chen, Dilip Krishnan, Dina Katabi, Phillip Isola, and Yonglong Tian. Scaling laws of synthetic images for model training ... for now. InCVPR, pages 7382–

  4. [12]

    Learning gen- erative visual models from few training examples: An incre- mental bayesian approach tested on 101 object categories

    Li Fei-Fei, Robert Fergus, and Pietro Perona. Learning gen- erative visual models from few training examples: An incre- mental bayesian approach tested on 101 object categories. CVIU, 106, 2007. 4

    2007

  5. [13]

    Clip-adapter: Better vision-language models with feature adapters

    Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. Clip-adapter: Better vision-language models with feature adapters. IJCV, 132, 2024. 2

    2024

  6. [14]

    DDGR: continual learning with deep diffusion-based generative replay

    Rui Gao and Weiwei Liu. DDGR: continual learning with deep diffusion-based generative replay. In ICML, pages 10744–10763, 2023. 3

    2023

  7. [15]

    Finetune like you pretrain: Im- proved finetuning of zero-shot vision models

    Sachin Goyal, Ananya Kumar, Sankalp Garg, Zico Kolter, and Aditi Raghunathan. Finetune like you pretrain: Im- proved finetuning of zero-shot vision models. In CVPR, pages 19338–19347, 2023. 2

    2023

  8. [16]

    Synthclip: Are we ready for a fully synthetic CLIP training? CoRR, abs/2402.01832, 2024

    Hasan Abed Al Kader Hammoud, Hani Itani, Fabio Pizzati, Philip Torr, Adel Bibi, and Bernard Ghanem. Synthclip: Are we ready for a fully synthetic CLIP training? CoRR, abs/2402.01832, 2024. 3

    2024 arXiv

  9. [17]

    EuroSAT: A novel dataset and deep learn- ing benchmark for land use and land cover classification

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. EuroSAT: A novel dataset and deep learn- ing benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observa- tions and Remote Sensing, 12, 2019. 4

    2019

  10. [18]

    Feedback-guided data synthesis for imbalanced classifica- tion

    Reyhane Askari Hemmat, Mohammad Pezeshki, Florian Bordes, Michal Drozdzal, and Adriana Romero-Soriano. Feedback-guided data synthesis for imbalanced classifica- tion. TMLR, 2024. 3

    2024

  11. [19]

    Learning a unified classifier incrementally via rebalancing

    Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning a unified classifier incrementally via rebalancing. In CVPR, pages 831–839, 2019. 2

    2019

  12. [20]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In ICLR, 2022. 2, 3

    2022

  13. [21]

    Dense network expansion for class in- cremental learning

    Zhiyuan Hu, Yunsheng Li, Jiancheng Lyu, Dashan Gao, and Nuno Vasconcelos. Dense network expansion for class in- cremental learning. In CVPR, pages 11858–11867, 2023. 2

    2023

  14. [22]

    Learn from downstream and be yourself in multimodal large language model fine-tuning

    Wenke Huang, Jian Liang, Zekun Shi, Didi Zhu, Guancheng Wan, He Li, Bo Du, Dacheng Tao, and Mang Ye. Learn from downstream and be yourself in multimodal large language model fine-tuning. CoRR, abs/2411.10928, 2024. 2

    2024 arXiv

  15. [23]

    Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc V . Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In ICML, 2021. 1, 2

    2021

  16. [24]

    Belongie, Bharath Hariharan, and Ser-Nam Lim

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge J. Belongie, Bharath Hariharan, and Ser-Nam Lim. Vi- sual prompt tuning. In ECCV, 2022. 2

    2022

  17. [25]

    Class-incremental learning using diffusion model for distillation and replay

    Quentin Jodelet, Xin Liu, Yin Jun Phua, and Tsuyoshi Mu- rata. Class-incremental learning using diffusion model for distillation and replay. In ICCVW, pages 3417–3425, 2023. 3

    2023

  18. [26]

    SDDGR: stable diffusion- based deep generative replay for class incremental object de- tection

    Junsu Kim, Hoseong Cho, Jihyeon Kim, Yihalem Yimolal Tiruneh, and Seungryul Baek. SDDGR: stable diffusion- based deep generative replay for class incremental object de- tection. In CVPR, pages 28772–28781, 2024. 3

    2024

  19. [27]

    Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A

    James Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Ku- maran, and Raia Hadsell. Overcoming catastrophic forget...

    2016 arXiv

  20. [28]

    3d object representations for fine-grained categorization

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In ICCVW, pages 554–561, 2013. 4

    2013

  21. [29]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. 1, 4

    2009

  22. [30]

    Continual classification learning using generative models

    Frantzeska Lavda, Jason Ramapuram, Magda Gregorova, and Alexandros Kalousis. Continual classification learning using generative models. CoRR, abs/1810.10612, 2018. 2 8

    2018 arXiv

  23. [31]

    Overcoming catastrophic forgetting by incremental moment matching

    Sang-Woo Lee, Jin-Hwa Kim, Jaehyun Jun, Jung-Woo Ha, and Byoung-Tak Zhang. Overcoming catastrophic forgetting by incremental moment matching. In NIPS, pages 4652– 4662, 2017. 2

    2017

  24. [32]

    What if we recaption billions of web images with llama-3? CoRR, abs/2406.08478, 2024

    Xianhang Li, Haoqin Tu, Mude Hui, Zeyu Wang, Bingchen Zhao, Junfei Xiao, Sucheng Ren, Jieru Mei, Qing Liu, Huangjie Zheng, Yuyin Zhou, and Cihang Xie. What if we recaption billions of web images with llama-3? CoRR, abs/2406.08478, 2024. 3

    2024 arXiv

  25. [33]

    Learning without forgetting

    Zhizhong Li and Derek Hoiem. Learning without forgetting. In ECCV, 2016. 2

    2016

  26. [34]

    SYNC- CLIP: synthetic data make CLIP generalize better in data- limited scenarios

    Mushui Liu, Weijie He, Ziqian Lu, and Yunlong Yu. SYNC- CLIP: synthetic data make CLIP generalize better in data- limited scenarios. CoRR, abs/2312.03805, 2023. 3

    2023 arXiv

  27. [35]

    Synthvlm: High-efficiency and high- quality synthetic data for vision language models

    Zheng Liu, Hao Liang, Xijie Huang, Wentao Xiong, Qin- han Yu, Linzhuang Sun, Chong Chen, Conghui He, Bin Cui, and Wentao Zhang. Synthvlm: High-efficiency and high- quality synthetic data for vision language models. CoRR, abs/2407.20756, 2024. 3

    2024 arXiv

  28. [36]

    Gradient episodic memory for continual learning

    David Lopez-Paz and Marc’Aurelio Ranzato. Gradient episodic memory for continual learning. In NIPS, pages 6467–6476, 2017. 2

    2017

  29. [37]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019. 4

    2019

  30. [38]

    Blaschko, and Andrea Vedaldi

    Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew B. Blaschko, and Andrea Vedaldi. Fine-grained visual classi- fication of aircraft. CoRR, abs/1306.5151, 2013. 4

    2013 arXiv

  31. [39]

    Diffclass: Diffusion-based class incremental learning

    Zichong Meng, Jie Zhang, Changdi Yang, Zheng Zhan, Pu Zhao, and Yanzhi Wang. Diffclass: Diffusion-based class incremental learning. In ECCV, 2024. 3

    2024

  32. [40]

    Improving multimodal datasets with image captioning

    Thao Nguyen, Samir Yitzhak Gadre, Gabriel Ilharco, Se- woong Oh, and Ludwig Schmidt. Improving multimodal datasets with image captioning. In NIPS, 2023. 3

    2023

  33. [41]

    Automated flower classification over a large number of classes

    Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In ICVGIP, pages 722–729, 2008. 4

    2008

  34. [42]

    Parkhi, Andrea Vedaldi, Andrew Zisserman, and C

    Omkar M. Parkhi, Andrea Vedaldi, Andrew Zisserman, and C. V . Jawahar. Cats and dogs. In CVPR, pages 3498–3505,

  35. [43]

    Ameya Prabhu, Philip H. S. Torr, and Puneet K. Dokania. Gdumb: A simple approach that questions our progress in continual learning. In ECCV, 2020. 2

    2020

  36. [44]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In ICML, 2021. 1, 2, 3

    2021

  37. [45]

    Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. icarl: Incremental classi- fier and representation learning. InCVPR, pages 5533–5542,

  38. [46]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, pages 10674– 10685, 2022. 1, 4

    2022

  39. [47]

    Imagenet large scale visual recognition challenge

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. IJCV, 115, 2015. 1

    2015

  40. [48]

    Continual learning with deep generative replay

    Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. Continual learning with deep generative replay. In NIPS, pages 2990–2999, 2017. 2

    2017

  41. [49]

    Is synthetic data all we need? benchmarking the robustness of models trained with synthetic images

    Krishnakant Singh, Thanush Navaratnam, Jannik Holmer, Simone Schaub-Meyer, and Stefan Roth. Is synthetic data all we need? benchmarking the robustness of models trained with synthetic images. In CVPRW, pages 2505–2515, 2024. 3

    2024

  42. [50]

    StableRep: Synthetic images from text-to- image models make strong visual representation learners

    Yonglong Tian, Lijie Fan, Phillip Isola, Huiwen Chang, and Dilip Krishnan. StableRep: Synthetic images from text-to- image models make strong visual representation learners. In NeurIPS, 2023

    2023

  43. [51]

    Learning vision from models rivals learning vision from data

    Yonglong Tian, Lijie Fan, Kaifeng Chen, Dina Katabi, Dilip Krishnan, and Phillip Isola. Learning vision from models rivals learning vision from data. In CVPR, pages 15887– 15898, 2024. 3

    2024

  44. [52]

    S-prompts learning with pre-trained transformers: An occam’s razor for domain incremental learning

    Yabin Wang, Zhiwu Huang, and Xiaopeng Hong. S-prompts learning with pre-trained transformers: An occam’s razor for domain incremental learning. In NIPS, 2022. 2

    2022

  45. [53]

    Dy, and Tomas Pfister

    Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen-Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer G. Dy, and Tomas Pfister. Dualprompt: Com- plementary prompting for rehearsal-free continual learning. In ECCV, 2022

    2022

  46. [54]

    Dy, and Tomas Pfister

    Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jen- nifer G. Dy, and Tomas Pfister. Learning to prompt for con- tinual learning. In CVPR, pages 139–149, 2022. 2

    2022

  47. [55]

    Robust fine-tuning of zero-shot models

    Mitchell Wortsman, Gabriel Ilharco, Mike Li, Jong Wook Kim, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, and Ludwig Schmidt. Robust fine-tuning of zero-shot models. CoRR, abs/2109.01903, 2021. 2, 4, 5

    2021 arXiv

  48. [56]

    Syn- thetic data is an elegant GIFT for continual vision-language models

    Bin Wu, Wuxuan Shi, Jinqiao Wang, and Mang Ye. Syn- thetic data is an elegant GIFT for continual vision-language models. In CVPR, 2025. 1, 3, 4, 5, 7, 2

    2025

  49. [57]

    Ehinger, Aude Oliva, and Antonio Torralba

    Jianxiong Xiao, James Hays, Krista A. Ehinger, Aude Oliva, and Antonio Torralba. SUN database: Large-scale scene recognition from abbey to zoo. In CVPR, pages 3485–3492,

  50. [58]

    DER: dynam- ically expandable representation for class incremental learn- ing

    Shipeng Yan, Jiangwei Xie, and Xuming He. DER: dynam- ically expandable representation for class incremental learn- ing. In CVPR, pages 3014–3023, 2021. 2

    2021

  51. [59]

    Fei Ye and Adrian G. Bors. Self-evolved dynamic expan- sion model for task-free continual learning. In ICCV, pages 22045–22055, 2023. 2

    2023

  52. [60]

    Controlled training data generation with diffusion mod- els

    Teresa Yeo, Andrei Atanov, Harold Benoit, Aleksandr Alek- seev, Ruchira Ray, Pooya Esmaeil Akhoondi, and Amir Za- mir. Controlled training data generation with diffusion mod- els. TMLR, 2025. 3

    2025

  53. [61]

    Lifelong learning with dynamically expandable net- works

    Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable net- works. In ICLR, 2018. 2 9

    2018

  54. [62]

    Coca: Contrastive captioners are image-text foundation models

    Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mo- jtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models. TMLR., 2022,

    2022

  55. [63]

    Boosting continual learning of vision-language models via mixture-of-experts adapters

    Jiazuo Yu, Yunzhi Zhuge, Lu Zhang, Ping Hu, Dong Wang, Huchuan Lu, and You He. Boosting continual learning of vision-language models via mixture-of-experts adapters. In CVPR, pages 23219–23230, 2024. 2, 4, 5

    2024

  56. [64]

    Florence: A new foundation model for computer vision

    Lu Yuan, Dongdong Chen, Yi-Ling Chen, Noel Codella, Xiyang Dai, Jianfeng Gao, Houdong Hu, Xuedong Huang, Boxin Li, Chunyuan Li, Ce Liu, Mengchen Liu, Zicheng Liu, Yumao Lu, Yu Shi, Lijuan Wang, Jianfeng Wang, Bin Xiao, Zhen Xiao, Jianwei Yang, Michael Zeng, Luowei Zhou, and Pe...

    2021 arXiv

  57. [65]

    Contin- ual learning through synaptic intelligence

    Friedemann Zenke, Ben Poole, and Surya Ganguli. Contin- ual learning through synaptic intelligence. In ICML, 2017. 2

    2017

  58. [66]

    Lit: Zero-shot transfer with locked-image text tuning

    Xiaohua Zhai, Xiao Wang, Basil Mustafa, Andreas Steiner, Daniel Keysers, Alexander Kolesnikov, and Lucas Beyer. Lit: Zero-shot transfer with locked-image text tuning. In CVPR, pages 18102–18112, 2022. 2

    2022

  59. [67]

    Preventing zero-shot transfer degradation in continual learning of vision-language models

    Zangwei Zheng, Mingyuan Ma, Kai Wang, Ziheng Qin, Xi- angyu Yue, and Yang You. Preventing zero-shot transfer degradation in continual learning of vision-language models. In ICCV, pages 19068–19079, 2023. 2

    2023

  60. [68]

    Preventing zero-shot transfer degradation in continual learning of vision-language models

    Zangwei Zheng, Mingyuan Ma, Kai Wang, Ziheng Qin, Xi- angyu Yue, and Yang You. Preventing zero-shot transfer degradation in continual learning of vision-language models. In ICCV, pages 19125–19136, 2023. 1, 4, 5, 2

    2023

  61. [69]

    Learning with- out forgetting for vision-language models

    Da-Wei Zhou, Yuanhan Zhang, Yan Wang, Jingyi Ning, Han-Jia Ye, De-Chuan Zhan, and Ziwei Liu. Learning with- out forgetting for vision-language models. TPAMI, pages 4489–4504, 2025. 2

    2025

  62. [70]

    Conditional prompt learning for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Zi- wei Liu. Conditional prompt learning for vision-language models. In CVPR, pages 16795–16804, 2022. 2

    2022

  63. [71]

    Learning to prompt for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. IJCV, 130, 2022. 2 10 LoRA-Loop: Closing the Synthetic Replay Cycle for Continual VLM Learning Supplementary Material

    2022

  64. [72]

    6 and Tab

    Detailed Results Tab. 6 and Tab. 7 present the Detailed Transfer, Avg., and Last metrics for different continual-training methods across the MTIL benchmark in Order I and Order II, respectively. These results highlight the ability of each method to adapt to new tasks while pre...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.