Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Bisecle: Binding and Separation in Continual Learning for Video Language Understanding

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Bisecle, a continual-learning method inspired by hippocampal binding and pattern separation, claims state-of-the-art accuracy and the lowest forgetting on NExT-QA, DramaQA, and STAR.

desk verdict Bisecle gets real gains on VideoQA continual learning, but the headline forgetting benefit flips on alternative task orders; worth a serious look but not a clean SOTA story. read the letter →

arxiv 2507.00469 v1 pith:XJAN4HBL submitted 2025-07-01 cs.CV cs.LG

Yue Tan , Xiaoqian Hu , Hao Xue , Celso De Melo , Flora D. Salim This is my paper
classification cs.CVcs.LG
keywords continuallearningvideoquestionansweringvision-languagemodelsparameter-efficientfine-tuningcatastrophicforgettingpromptcontrastivehippocampus-inspiredAI
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that catastrophic forgetting and update conflicts in parameter-efficient continual video question answering can be sharply reduced by mimicking two hippocampus mechanisms: rapid binding and pattern separation. It packages these into Bisecle, a method that adds auxiliary question-generation and video-generation supervision alongside the usual answer prediction, and that regularizes shared learnable prompts with a contrastive loss against task-type embeddings. On NExT-QA, DramaQA, and STAR, Bisecle reports new best average accuracy and the lowest forgetting among compared methods, with the largest gain on NExT-QA (62.37 accuracy versus 55.14 for the previous LLM-based approach). If the results hold, lightweight continual adaptation of frozen vision-language models becomes substantially more reliable, which matters for streaming video applications such as wearable-camera assistants.

What carries the argument

The central machinery is a two-part objective that translates two hippocampal principles into loss terms. The binding side is multi-directional supervision: in addition to $L_A = -\log P(A \mid V, Q)$, the model minimizes $L_Q = -\log P(Q \mid V, A)$ and $L_V = -\log P(V \mid Q, A)$, where the video term is computed by a softmax over frame-feature similarities so that each training sample stores associations among all three modalities rather than one direction only. The separation side is contrastive prompt learning: a single shared prompt matrix $P$ is injected into multiple transformer layers, and a per-task learnable embedding $e_t$ plays the role of a non-overlapping subspace; the reweighted prompt $\hat{p}_t^i = (q_t^i \cdot P^\top) \cdot P$ is pushed toward $e_t$ and away from other task embeddings by an InfoNCE-style loss $L_P$ with temperature $\tau$. This is what carries the argument: the auxiliary losses are claimed to prevent forgetting by deepening cross-modal binding, and the contrastive alignment is claimed to prevent update conflict by keeping task knowledge separated in prompt space.

What would settle it

A single experiment would settle the matter: run Bisecle in a task-free continual setting where examples arrive one at a time without task labels, using the same NExT-QA split, and check whether the contrastive prompt loss can still isolate knowledge without supervision from task-type embeddings; if accuracy and forgetting revert to backbone levels, the claimed gains depend on the known-task-boundary protocol rather than on the binding and separation mechanisms themselves.

Watch

Extended reading notes

Core claim

Bisecle is a learning objective plus a prompt-regularization strategy. The objective augments the standard answer-prediction loss $L_A$ with two auxiliary tasks: a question-prediction loss $L_Q$ that trains the frozen-LLM adapter to reconstruct the question from video and answer tokens, and a video-prediction loss $L_V$ that trains it to reconstruct visual tokens from question and answer via a softmax over frame-feature similarities, following a mutual-information objective. These bind the three elements of each VideoQA sample in all directions, so knowledge is not stored only along the question-to-answer pathway. The regularization strategy keeps one shared, task-aware prompt matrix $P$ across all tasks but learns a small task-type embedding $e_t$ per task and applies a contrastive loss $L_P$ that pulls the question-reweighted prompt toward its own task embedding and away from others, isolating task-specific knowledge in separate latent regions. The total loss is $L = L_A + L_Q + L_V + \gamma L_P$, and the paper argues that this explains its reported results: 62.37 accuracy with 5.34 forgetting on NExT-QA, 71.49 with 10.37 on DramaQA, and 52.16 with 7.60 on STAR, beating six prior continual-learning methods, with the ablation attributing the largest single gain to $L_Q$.

Load-bearing premise

The central claim rests on the assumption that splitting each benchmark into question-type tasks, presented in a fixed order with the task identity known during training, faithfully represents continual learning for real evolving video streams, an assumption the authors themselves flag as an open challenge.

Editorial extensions

If this is right

  • Continual VideoQA on frozen multimodal LLMs is feasible with much smaller forgetting than previously reported: on NExT-QA the forgetting metric drops from 7.43 (ColPro) to 5.34 while accuracy rises by 7.23 points.
  • Parameter-efficient continual learning can be driven by auxiliary reconstruction objectives rather than rehearsal buffers or per-task parameter allocation, reducing memory overhead.
  • The gains hold across three benchmarks with different question taxonomies (8 task types on NExT-QA, 5 on DramaQA, 4 on STAR), indicating the mechanisms are not tuned to a single dataset.
  • Increasing the number of prompt-injection layers from 8 to 32 improves both accuracy and forgetting resistance, so the method scales with prompt capacity.
  • The design transfers across backbone scale: experiments with 1B, 7B, and 13B LLaMA variants show accuracy gains on each, with the largest forgetting reduction at mid-scale.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the question-prediction loss contributes the largest gain, a natural testable extension is whether substituting other reconstruction directions, such as captioning, future-frame prediction, or mask-token recovery, yields similar forgetting resistance without the video-token softmax approximation.
  • The known-task-boundary assumption is the paper's own stated limitation; a task-free variant that infers task identity from the input or drops task-type embeddings would determine whether the contrastive mechanism can survive in truly open-ended streams.
  • The paper's qualitative comparisons with API-only frontier models suggest that Bisecle's advantage lies in delayed-causality and location-sequencing reasoning; one could probe this directly by benchmarking Bisecle against those models on tasks engineered for exactly those abilities.
  • If the mechanism generalizes, the same binding-plus-separation objective could transfer to audio-visual or embodied continual learning, where the modality graph has more than two edges; the paper mentions this as future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. Bisecle proposes a parameter-efficient continual learning framework for video question answering built on a frozen LLaMA-Adapter backbone. It introduces two hippocampus-inspired modules: multi-directional supervision, which adds question-prediction (L_Q) and video-prediction (L_V) objectives to the standard answer-generation objective, and contrastive prompt learning, which trains shared task-aware prompts to align with learned task-type embeddings through a contrastive loss. The method is evaluated on NExT-QA, DramaQA, and STAR under the ColPro task-splitting protocol, and the paper reports improved average accuracy and reduced average forgetting relative to six baselines, along with ablations, data-efficiency experiments, LLM-size scaling, and alternatives to the proposed contrastive formulation.

Significance. If the reported gains are reliable, Bisecle is a useful contribution to continual VideoQA: the idea of adding auxiliary cross-modal objectives and contrastive prompt separation is simple and plausible, and the empirical evidence shows consistent improvements across three benchmarks and in most ablations, including when the backbone and LLM size are varied. The paper should be credited for reporting an alternative task-order study (Appendix B.2), a comparison with API-based frontier models (Appendix B.4), and an explicit statement of limitations (Appendix C). The main reservation is that the signature forgetting-mitigation claim is not stable across task orders, and the experimental protocol lacks statistical rigor.

major comments (3)
  1. [Appendix B.2, Table 10] Table 10 undermines the headline claim that Bisecle robustly mitigates catastrophic forgetting. On the orders <TP, TN, CH, TC, DL, DO, CW, DC> and <DO, CW, DC, CH, TP, TC, TN, DL>, Bisecle has higher Avg. Fog (7.16 and 8.93) than the LLaMA-Adapter backbone (5.89 and 7.15), despite higher accuracy. Because the introduction and abstract motivate Bisecle with streaming data that has no canonical task order, and because the main results in Table 1 use only the single order inherited from ColPro, the central forgetting-mitigation claim is order-dependent. The phrase "in most cases" in B.2 implicitly concedes this. The authors should either restrict the claim to the favorable orders, or provide an order-averaged analysis with variances and an explanation of why forgetting can increase while accuracy increases; otherwise the claimed robustness is not established.
  2. [Section 4.1 and A.1.4, Tables 1–3] All experiments are run with a single random seed (A.1.4: "fix the random seed to 0"), and no error bars or significance tests are reported. Given that hyperparameters are tuned per dataset (batch size, weight decay, token truncation, contrastive loss weight), the differences in Tables 1–3 could be due to run-to-run variation rather than the method. At minimum, the authors should give mean and standard deviation over several seeds (e.g., 3–5) for the main comparison and ablations, and specify how hyperparameters were chosen for baselines and Bisecle.
  3. [Appendix C, Section 1] The framing of the problem as adapting to "continuously evolving data streams" is only partially matched by the evaluation protocol. As Appendix C states, the framework assumes task boundaries are known during training, and the task ordering is fixed; no experiment addresses a task-free or online stream. This does not invalidate the within-protocol results, but the conclusions about open-world streaming generalization in the abstract and Section 1 should be softened, or the protocol should be extended with blurry or free task boundaries to support the stronger claims.
minor comments (5)
  1. [Section 3.2, Eq. (3)] Equation (3) is described as maximizing mutual information and is framed in the spirit of InfoNCE, but it is a token-level softmax over the current video's visual tokens with no temperature and no explicit positive/negative pair construction. The connection to mutual information should be made explicit, or the wording should be changed to avoid overclaiming the theoretical grounding of L_V.
  2. [Section 3.2 and Figure 3] The introduction says the method can "generate videos from questions and answers," but the actual video prediction objective in Eq. (3) is a discriminative classification of frame tokens, not a generative model of videos; this wording is misleading.
  3. [Section 3.3, Eq. (4)] The notation for the task type embedding is inconsistent: e_t is called a vector, then a matrix, and the denominator A(t) = {0, ..., t} uses an index set whose starting point is ambiguous when tasks are indexed from 1 to T. Please clarify the dimensions and indexing.
  4. [Throughout] There are a few typos, such as "casual" for "causal" in the Figure 3 caption and in Section 3.2; a careful proofreading pass is needed.
  5. [Reproducibility] No code or checkpoint is provided, which makes the experiments difficult to reproduce; providing code or at least detailed per-dataset configurations for all baselines would strengthen the paper.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the reported metrics are defined on external benchmarks and are not constructed from Bisecle's fitted parameters or auxiliary losses.

full rationale

Bisecle's claimed gains are measured by Avg. Acc and Avg. Fog on NExT-QA, DramaQA, and STAR against external baselines; none of these metrics is computed from the multi-directional supervision or contrastive-prompt losses. The auxiliary losses LQ (Eq. 2), LV (Eq. 3), and LP (Eq. 4) are training regularizers over labeled triples (video, question, answer); their learnable task-type embeddings and prompt weights influence the trained parameters but are not reused as test-time predictions. The fixed task order and task boundaries are inherited from ColPro [9], an external prior work, rather than from a self-citation chain. The only citation with overlapping authors is [6] (VILCO-Bench, co-authored by Xue, De Melo, and Salim), and it appears in Related Work as benchmark context; it is not load-bearing for the method's derivation. The paper's own limitations (Appendix C: task boundaries are assumed known; Appendix B.2: forgetting gains are task-order dependent) are empirical robustness and scoping concerns, not evidence that a fitted input is being renamed as a prediction. The 'video prediction' loss in Eq. 3 is implemented as an intra-video alignment contrast rather than a true marginal likelihood, but this is a modeling and fidelity concern and does not make the evaluation circular.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method is empirical and adds no new physical or formal entities. Its central claim rests on three domain choices: the frozen-adapter backbone, the question-type task protocol, and the InfoNCE surrogate for video prediction. The listed hyperparameters are tuned per dataset and therefore carry part of the performance.

free parameters (5)
  • Number of prompt tokens and prompt injection layers (Np, Lp) = Prompt length 10; # P. Layer swept from 8 to 32, final 32
    The number of learnable prompt tokens per layer and the number of transformer layers with injected prompts are chosen by the authors; Table 4 shows accuracy improves with more prompt layers, so the final choice affects results.
  • Contrastive temperature tau = 1.28 (NExT-QA), 1.25 (DramaQA and STAR)
    Per-dataset temperature hyperparameter in the contrastive loss of Eq. (4); tuned separately for each benchmark.
  • Contrastive loss weight gamma = 0.15 (NExT-QA), 0.10 (DramaQA and STAR)
    Trade-off hyperparameter in the total loss L = LA + LQ + LV + gamma * LP; set per dataset.
  • Task type embedding matrix e_t = Sizes [8,4096], [5,4096], [4,4096] for the three datasets
    Learned task-specific embeddings optimized on each benchmark; the contrastive loss directly aligns prompts with these fitted vectors, so their values are part of the method.
  • Training configuration (learning rate, batch size, epochs, weight decay, token truncation) = lr 0.09; batch 32/4/16; 5 epochs with 2 warmup; weight decay 0.14/0.10; truncation 128/280/150
    These standard training choices vary per dataset and affect reported numbers; they are part of the unreleased experimental configuration.
assumptions (4)
  • domain assumption The frozen LLaMA-Adapter with a trainable projection layer and injected prompts is a sufficient parameter-efficient backbone for VideoQA continual learning.
    Invoked in Sections 3.1 and 4.1; the method's gains are measured on top of this backbone, not on a general VLM.
  • domain assumption Partitioning NExT-QA, DramaQA, and STAR into question-type tasks with fixed task orders faithfully represents continual learning for video streams.
    Set in Section 4.1 following ColPro; if these splits understate or distort real distribution shift, the claimed cross-task generalization is not established.
  • domain assumption The InfoNCE objective in Eq. (3) is a valid surrogate for the conditional probability P(V|Q,A) and provides useful cross-modal binding signal.
    Section 3.2 replaces direct visual generation with a contrastive loss over existing visual tokens; the paper does not prove this surrogate estimates the stated generative objective.
  • domain assumption Task identity is available during training so task type embeddings can be assigned per task.
    Stated in Limitations Section C; the contrastive prompt learning module requires known task boundaries at training time.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bisecle: Binding and Separation in Continual Learning for Video Language Understanding." pith.science (2026). https://pith.science/paper/XJAN4HBL

@misc{pith2026250700469,
  author       = {Pith},
  title        = {Pith review of: Bisecle: Binding and Separation in Continual Learning for Video Language Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XJAN4HBL}},
  note         = {Machine review of arXiv:2507.00469}
}
read the original abstract

Frontier vision-language models (VLMs) have made remarkable improvements in video understanding tasks. However, real-world videos typically exist as continuously evolving data streams (e.g., dynamic scenes captured by wearable glasses), necessitating models to continually adapt to shifting data distributions and novel scenarios. Considering the prohibitive computational costs of fine-tuning models on new tasks, usually, a small subset of parameters is updated while the bulk of the model remains frozen. This poses new challenges to existing continual learning frameworks in the context of large multimodal foundation models, i.e., catastrophic forgetting and update conflict. While the foundation models struggle with parameter-efficient continual learning, the hippocampus in the human brain has evolved highly efficient mechanisms for memory formation and consolidation. Inspired by the rapid Binding and pattern separation mechanisms in the hippocampus, in this work, we propose Bisecle for video-language continual learning, where a multi-directional supervision module is used to capture more cross-modal relationships and a contrastive prompt learning scheme is designed to isolate task-specific knowledge to facilitate efficient memory storage. Binding and separation processes further strengthen the ability of VLMs to retain complex experiences, enabling robust and efficient continual learning in video understanding tasks. We perform a thorough evaluation of the proposed Bisecle, demonstrating its ability to mitigate forgetting and enhance cross-task generalization on several VideoQA benchmarks.

Figures

Figures reproduced from arXiv: 2507.00469 by the authors.

Figure 1
Figure 1. (a) The backbone of our continual learning framework for video understanding; (b) The [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. A “Bisecle” with two wheels (binding and sepa￾ration) running on the road (a sequence of tasks). To answer the above question, in this paper, we propose a simple yet effective method, Bisecle, inspired by rapid Binding and pattern separation mechanisms, for video-language continual learning (see [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The sketch map to explain the two modules in Bisecle, multi-directional supervision and [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Performance on different sizes of training datasets [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Answers by Backbone and Bisecle. Case Study. We further evaluate the effectiveness of Bisecle by examin￾ing failure cases (See [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: (a) and (b): t-SNE visualization of learnable [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]
Figure 7
Figure 7. Figure 7: More cases on NExT-QA. Question: Which object was taken by the person? … Backbone: A. The bag Bisecle (ours): D. The sandwich Choices: A. The bag B. The sandwich Question: Which object was taken by the person? Backbone: The bag Bisecle (ours): The sandwich STAR case st…
Figure 8
Figure 8. Figure 8: More cases on STAR. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: The performance of Gemini 2.0 flash, GPT-4o-mini, LLaMA-Adapter, and our method [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]
Figure 10
Figure 10. Figure 10: The performance of Gemini 2.0 Flash on different tasks with various numbers of input [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]
Figure 11
Figure 11. Figure 11: The performance of GPT-4o and GPT-4o-mini on the learning tasks of NExT-QA in our [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]
Figure 12
Figure 12. Figure 12: The weakness of frontier VLMs in dealing with continual learning tasks can be found as [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Text as Partial Constraint: Core-Residual Alignment for Robust Vision-Language Learning

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Aligning images to multi-view caption cores while suppressing orthogonal residual text and disagreement-aware temperature improves robust zero-shot recognition and LVLM transfer.

Reference graph

Works this paper leans on

62 extracted references · 42 canonical work pages · cited by 1 Pith paper

  1. [1]

    Video understanding with large language models: A survey

    Yunlong Tang, Jing Bi, Siting Xu, Luchuan Song, Susan Liang, Teng Wang, Daoan Zhang, Jie An, Jingyang Lin, Rongyi Zhu, et al. Video understanding with large language models: A survey. arXiv preprint arXiv:2312.17432, 2023

    arXiv 2023

  2. [2]

    Streaming long video understanding with large language models

    Rui Qian, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Shuangrui Ding, Dahua Lin, and Jiaqi Wang. Streaming long video understanding with large language models. Advances in Neural Information Processing Systems, 37:119336–119360, 2023

    work page 2023

  3. [3]

    Video-chatgpt: Towards detailed video understanding via large vision and language models

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 12585–12602, 2024

    work page 2024

  4. [4]

    Rethinking video anomaly detection-a continual learning approach

    Keval Doshi and Yasin Yilmaz. Rethinking video anomaly detection-a continual learning approach. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 3961–3970, 2022

    work page 2022

  5. [5]

    Pivot: Prompting for video contin- ual learning

    Andrés Villa, Juan León Alcázar, Motasem Alfarra, Kumail Alhamoud, Julio Hurtado, Fabian Caba Heilbron, Alvaro Soto, and Bernard Ghanem. Pivot: Prompting for video contin- ual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24214–24223, 2023

    work page 2023

  6. [6]

    Vilco-bench: Video language continual learning benchmark

    Tianqi Tang, Shohreh Deldari, Hao Xue, Celso De Melo, and Flora Salim. Vilco-bench: Video language continual learning benchmark. Advances in Neural Information Processing Systems, 37:70213–70229, 2024

    work page 2024

  7. [7]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems , 35:23716–23736, 2022

    2022

  8. [8]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730–19742. PMLR, 2023

    2023

Show all 62 references
  1. [9]

    Empowering large language model for continual video question answering with collaborative prompting

    Chen Cai, Zheng Wang, Jianjun Gao, Wenyang Liu, Ye Lu, Runzhong Zhang, and Kim-Hui Yap. Empowering large language model for continual video question answering with collaborative prompting. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processin...

    2024

  2. [10]

    An empirical study of catastrophic forgetting in large language models during continual fine-tuning

    Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. arXiv preprint arXiv:2308.08747, 2023

    2023 arXiv

  3. [11]

    Llama-adapter: Efficient fine-tuning of language models with zero-init attention

    Renrui Zhang, Jiaming Han, Chris Liu, Peng Gao, Aojun Zhou, Xiangfei Hu, Shilin Yan, Pan Lu, Hongsheng Li, and Yu Qiao. Llama-adapter: Efficient fine-tuning of language models with zero-init attention. arXiv preprint arXiv:2303.16199, 2023

    2023 arXiv

  4. [12]

    Impaired auditory recognition memory in amnesic patients with medial temporal lobe lesions

    Larry R Squire, Heike Schmolck, and Shauna M Stark. Impaired auditory recognition memory in amnesic patients with medial temporal lobe lesions. Learning & Memory, 8(5):252–256, 2001

    2001

  5. [13]

    Anatomy of the hippocampal formation

    Christian Schultz and Maren Engelhardt. Anatomy of the hippocampal formation. Frontiers of neurology and neuroscience, 34:6–17, 2014

    2014

  6. [14]

    The hippocampus supports multiple cognitive processes through relational binding and comparison

    Rosanna K Olsen, Sandra N Moses, Lily Riggs, and Jennifer D Ryan. The hippocampus supports multiple cognitive processes through relational binding and comparison. Frontiers in human neuroscience, 6:146, 2012

    2012

  7. [15]

    Neural mechanisms of binding in the hippocampus and neocortex: insights from computational models., 2006

    Daniel M Cer and Randall C O’Reilly. Neural mechanisms of binding in the hippocampus and neocortex: insights from computational models., 2006

    2006

  8. [16]

    Pattern separation in the hippocampus

    Michael A Yassa and Craig EL Stark. Pattern separation in the hippocampus. Trends in neurosciences, 34(10):515–525, 2011

    2011

  9. [17]

    Pattern separation in the dentate gyrus and ca3 of the hippocampus

    Jill K Leutgeb, Stefan Leutgeb, May-Britt Moser, and Edvard I Moser. Pattern separation in the dentate gyrus and ca3 of the hippocampus. science, 315(5814):961–966, 2007

    2007

  10. [18]

    Trace: A comprehensive benchmark for continual learning in large language models

    Xiao Wang, Yuansen Zhang, Tianze Chen, Songyang Gao, Senjie Jin, Xianjun Yang, Zhiheng Xi, Rui Zheng, Yicheng Zou, Tao Gui, et al. Trace: A comprehensive benchmark for continual learning in large language models. arXiv preprint arXiv:2310.06762, 2023

    arXiv

  11. [19]

    Towards lifelong learning of large language models: A survey

    Junhao Zheng, Shengjie Qiu, Chengming Shi, and Qianli Ma. Towards lifelong learning of large language models: A survey. ACM Computing Surveys, 57(8):1–35, 2025

    2025

  12. [20]

    Recent advances of foundation language models-based continual learning: A survey

    Yutao Yang, Jie Zhou, Xuanwen Ding, Tianyu Huai, Shunyu Liu, Qin Chen, Yuan Xie, and Liang He. Recent advances of foundation language models-based continual learning: A survey. ACM Computing Surveys, 57(5):1–38, 2025

    2025

  13. [21]

    Ernie 2.0: A continual pre-training framework for language understanding

    Yu Sun, Shuohuan Wang, Yukun Li, Shikun Feng, Hao Tian, Hua Wu, and Haifeng Wang. Ernie 2.0: A continual pre-training framework for language understanding. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 8968–8975, 2020

    2020

  14. [22]

    Continual pre-training mitigates forgetting in language and vision

    Andrea Cossu, Antonio Carta, Lucia Passaro, Vincenzo Lomonaco, Tinne Tuytelaars, and Davide Bacciu. Continual pre-training mitigates forgetting in language and vision. Neural Networks, 179:106492, 2024

    2024

  15. [23]

    Do pre-trained models benefit equally in continual learning? In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 6485–6493, 2023

    Kuan-Ying Lee, Yuanyi Zhong, and Yu-Xiong Wang. Do pre-trained models benefit equally in continual learning? In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 6485–6493, 2023

    2023

  16. [24]

    Continual pre-training of language models

    Zixuan Ke, Yijia Shao, Haowei Lin, Tatsuya Konishi, Gyuhak Kim, and Bing Liu. Continual pre-training of language models. arXiv preprint arXiv:2302.03241, 2023

    2023 arXiv

  17. [25]

    Continual instruction tuning for large multimodal models

    Jinghan He, Haiyun Guo, Ming Tang, and Jinqiao Wang. Continual instruction tuning for large multimodal models. arXiv preprint arXiv:2311.16206, 2023

    2023 arXiv

  18. [26]

    Coin: A benchmark of continual instruction tuning for multimodel large language models

    Cheng Chen, Junchen Zhu, Xu Luo, Hengtao Shen, Jingkuan Song, and Lianli Gao. Coin: A benchmark of continual instruction tuning for multimodel large language models. Advances in Neural Information Processing Systems, 37:57817–57840, 2024

    2024

  19. [27]

    Continual llava: Continual instruction tuning in large vision-language models

    Meng Cao, Yuyang Liu, Yingfei Liu, Tiancai Wang, Jiahua Dong, Henghui Ding, Xiangyu Zhang, Ian Reid, and Xiaodan Liang. Continual llava: Continual instruction tuning in large vision-language models. arXiv preprint arXiv:2411.02564, 2024. 11

    2024 arXiv

  20. [28]

    Copf: Continual learning human preference through optimal policy fitting

    Han Zhang, Lin Gui, Yuanzhao Zhai, Hui Wang, Yu Lei, and Ruifeng Xu. Copf: Continual learning human preference through optimal policy fitting. CoRR, 2023

    2023

  21. [29]

    Continual learning for instruction following from realtime feedback

    Alane Suhr and Yoav Artzi. Continual learning for instruction following from realtime feedback. Advances in Neural Information Processing Systems, 36:32340–32359, 2023

    2023

  22. [30]

    Slca: Slow learner with classifier alignment for continual learning on a pre-trained model

    Gengwei Zhang, Liyuan Wang, Guoliang Kang, Ling Chen, and Yunchao Wei. Slca: Slow learner with classifier alignment for continual learning on a pre-trained model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19148–19158, 2023

    2023

  23. [31]

    Continual learning for large language models: A survey

    Tongtong Wu, Linhao Luo, Yuan-Fang Li, Shirui Pan, Thuy-Trang Vu, and Gholamreza Haffari. Continual learning for large language models: A survey. arXiv preprint arXiv:2402.01364, 2024

    2024 arXiv

  24. [32]

    Fine-tuned language models are continual learners

    Thomas Scialom, Tuhin Chakrabarty, and Smaranda Muresan. Fine-tuned language models are continual learners. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6107–6122, 2022

    2022

  25. [33]

    Inscl: A data-efficient continual learning paradigm for fine-tuning large language models with instructions

    Yifan Wang, Yafei Liu, Chufan Shi, Haoling Li, Chen Chen, Haonan Lu, and Yujiu Yang. Inscl: A data-efficient continual learning paradigm for fine-tuning large language models with instructions. In Proceedings of the 2024 Conference of the North American Chapter of the Associat...

    2024

  26. [34]

    Large- scale lifelong learning of in-context instructions and how to tackle it

    Jisoo Mok, Jaeyoung Do, Sungjin Lee, Tara Taghavi, Seunghak Yu, and Sungroh Yoon. Large- scale lifelong learning of in-context instructions and how to tackle it. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) ...

    2023

  27. [35]

    Progressive prompts: Continual learning for language models

    Anastasia Razdaibiedina, Yuning Mao, Rui Hou, Madian Khabsa, Mike Lewis, and Amjad Almahairi. Progressive prompts: Continual learning for language models. In The Eleventh International Conference on Learning Representations, 2023

    2023

  28. [36]

    Exploring the benefits of training expert language models over instruction tuning

    Joel Jang, Seungone Kim, Seonghyeon Ye, Doyoung Kim, Lajanugen Logeswaran, Moontae Lee, Kyungjae Lee, and Minjoon Seo. Exploring the benefits of training expert language models over instruction tuning. In International Conference on Machine Learning, pages 14702–14729. PMLR, 2023

    2023

  29. [37]

    Symbolic replay: Scene graph as prompt for continual learning on vqa task

    Stan Weixian Lei, Difei Gao, Jay Zhangjie Wu, Yuxuan Wang, Wei Liu, Mengmi Zhang, and Mike Zheng Shou. Symbolic replay: Scene graph as prompt for continual learning on vqa task. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 1250–1259, 2023

    2023

  30. [38]

    Decouple before interact: Multi-modal prompt learning for continual visual question answering

    Zi Qian, Xin Wang, Xuguang Duan, Pengda Qin, Yuhong Li, and Wenwu Zhu. Decouple before interact: Multi-modal prompt learning for continual visual question answering. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2953–2962, 2023

    2023

  31. [39]

    Just a glimpse: Rethinking temporal information for video continual learning

    Lama Alssum, Juan Leon Alcazar, Merey Ramazanova, Chen Zhao, and Bernard Ghanem. Just a glimpse: Rethinking temporal information for video continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2474–2483, 2023

    2023

  32. [40]

    Continual learning of unsupervised monocular depth from videos

    Hemang Chawla, Arnav Varma, Elahe Arani, and Bahram Zonooz. Continual learning of unsupervised monocular depth from videos. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 8419–8429, 2024

    2024

  33. [41]

    A baseline on continual learning methods for video action recognition

    Giulia Castagnolo, Concetto Spampinato, Francesco Rundo, Daniela Giordano, and Simone Palazzo. A baseline on continual learning methods for video action recognition. In 2023 IEEE International Conference on Image Processing (ICIP), pages 3240–3244. IEEE, 2023

    2023

  34. [42]

    Class-incremental learning for action recognition in videos

    Jaeyoo Park, Minsoo Kang, and Bohyung Han. Class-incremental learning for action recognition in videos. In Proceedings of the IEEE/CVF international conference on computer vision, pages 13698–13707, 2021. 12

    2021

  35. [43]

    Dam: Dynamic adapter merging for continual video qa learning

    Feng Cheng, Ziyang Wang, Yi-Lin Sung, Yan-Bo Lin, Mohit Bansal, and Gedas Bertasius. Dam: Dynamic adapter merging for continual video qa learning. arXiv preprint arXiv:2403.08755, 2024

    2024 arXiv

  36. [44]

    Continual predictive learning from videos

    Geng Chen, Wendong Zhang, Han Lu, Siyu Gao, Yunbo Wang, Mingsheng Long, and Xiaokang Yang. Continual predictive learning from videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10728–10737, 2022

    2022

  37. [45]

    Vqacl: A novel visual question answering continual learning setting

    Xi Zhang, Feifei Zhang, and Changsheng Xu. Vqacl: A novel visual question answering continual learning setting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19102–19112, 2023

    2023

  38. [46]

    A survey on vision transformer

    Kai Han, Yunhe Wang, Hanting Chen, Xinghao Chen, Jianyuan Guo, Zhenhua Liu, Yehui Tang, An Xiao, Chunjing Xu, Yixing Xu, et al. A survey on vision transformer. IEEE transactions on pattern analysis and machine intelligence, 45(1):87–110, 2022

    2022

  39. [47]

    Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting

    Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher, and Caiming Xiong. Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting. In International conference on machine learning, pages 3925–3934. PMLR, 2019

    2019

  40. [48]

    Overcoming catastrophic forgetting for continual learning via model adaptation

    Wenpeng Hu, Zhou Lin, Bing Liu, Chongyang Tao, Zhengwei Tao Tao, Dongyan Zhao, Jin- wen Ma, and Rui Yan. Overcoming catastrophic forgetting for continual learning via model adaptation. In International conference on learning representations, 2019

    2019

  41. [49]

    The hippocampal formation as a hierarchical generative model supporting generative replay and continual learning

    Ivilin Stoianov, Domenico Maisto, and Giovanni Pezzulo. The hippocampal formation as a hierarchical generative model supporting generative replay and continual learning. Progress in Neurobiology, 217:102329, 2022

    2022

  42. [50]

    Continual learning with deep generative replay

    Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. Continual learning with deep generative replay. Advances in neural information processing systems, 30, 2017

    2017

  43. [51]

    Expe- rience replay for continual learning

    David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Expe- rience replay for continual learning. Advances in neural information processing systems, 32, 2019

    2019

  44. [52]

    Large language models are temporal and causal reasoners for video question answering

    Dohwan Ko, Ji Lee, Woo-Young Kang, Byungseok Roh, and Hyunwoo Kim. Large language models are temporal and causal reasoners for video question answering. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4300–4316, 2023

    2023

  45. [53]

    Learning to prompt for continual learning

    Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 139–149, 2022

    2022

  46. [54]

    Next-qa: Next phase of question- answering to explaining temporal actions

    Junbin Xiao, Xindi Shang, Angela Yao, and Tat-Seng Chua. Next-qa: Next phase of question- answering to explaining temporal actions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9777–9786, 2021

    2021

  47. [55]

    Dramaqa: Character-centered video story understanding with hierarchical qa

    Seongho Choi, Kyoung-Woon On, Yu-Jung Heo, Ahjeong Seo, Youwon Jang, Minsu Lee, and Byoung-Tak Zhang. Dramaqa: Character-centered video story understanding with hierarchical qa. In Proceedings of the aaai conference on artificial intelligence, volume 35, pages 1166–1174, 2021

    2021

  48. [56]

    Star: A benchmark for situated reasoning in real-world videos

    Bo Wu, Shoubin Yu, Zhenfang Chen, Joshua B Tenenbaum, and Chuang Gan. Star: A benchmark for situated reasoning in real-world videos. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021

    2021

  49. [57]

    Dualprompt: Complementary prompting for rehearsal-free continual learning

    Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen-Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, et al. Dualprompt: Complementary prompting for rehearsal-free continual learning. In European conference on computer vision, pages 631–648. Springe...

    2022

  50. [58]

    A unified continual learning framework with general parameter-efficient tuning

    Qiankun Gao, Chen Zhao, Yifan Sun, Teng Xi, Gang Zhang, Bernard Ghanem, and Jian Zhang. A unified continual learning framework with general parameter-efficient tuning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11483–11493, 2023

    2023

  51. [59]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  52. [60]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, G Heigold, S Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on...

    2020

  53. [61]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pa...

    2021

  54. [62]

    general” instructions and E-Prompts for encoding task-specific “expert

    Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008. 14 A Experimental Details A.1 Detailed Setting of Table 1 A.1.1 Task and Data Setting For NExT-QA and DramaQA datasets, we follow the continual learning...

    2008

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.