Pith. sign in

REVIEW 4 major objections 6 minor 86 references

Progressive Homeostatic and Plastic Prompt Tuning for Audio-Visual Multi-Task Incremental Learning

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a three-stage progressive prompt hierarchy lets one model learn four audio-visual tasks in sequence, beat all seven baselines on anti-forgetting, and become the only compared method with positive cross-task transfer…

desk verdict A genuinely new problem setting and a lot of careful experimentation, but the paper's central three-stage claim is contradicted by its own ablations, and the custom transfer metric behaves opposite to its stated purpose. read the letter →

arxiv 2507.21588 v1 pith:FICZBTQH submitted 2025-07-29 cs.AI cs.CV

Jiong Yin , Liang Li , Jiehua Zhang , Yuhan Gao , Chenggang Yan , Xichun Sheng This is my paper
classification cs.AIcs.CV
keywords audio-visualmulti-taskincrementallearningcatastrophicforgettingprompttuningcontinualknowledgetransferdynamicgenerationeventlocalizationquestionanswering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes PHP, a three-stage prompting method for learning an ongoing stream of audio-visual tasks without retraining on old data. Its central claim is that the forgetting-versus-transfer tradeoff can be resolved by progressive specialization through network depth: shared cross-modal representations in shallow layers, task-specific but modality-shared prompts in middle layers, and fully task-and-modality-specific prompts in deep layers. On four tasks (AVE, AVVP, AVS, AVQA), PHP reports the best anti-forgetting scores among the compared methods and is the only one showing positive multi-task transfer, quantified as $+7.79\%$ on the paper's Diff metric.

What carries the argument

The load-bearing mechanism is the progressive ordering of three prompt components across the depth of frozen CLIP and CLAP backbones, expressed as the shallow-middle-deep (S-M-D) principle: Task-shared Modality Aggregating (TMA) adapters at shallow layers perform channel, spatial, and temporal cross-modal attention that is shared by all tasks; Task-specific Modality-shared Dynamic Generating (TMDG) adapters at middle layers select and generate instance-level prompts from a learned prompt pool via self-attention, keeping prompts task-specific but consistent across audio and video; Task-specific Modality-Independent (TMI) prompts at deep layers attach separate per-task, per-modality tokens. The transfer claim itself is carried by the paper's bespoke metric Diff, defined as $\frac{A_{\mathrm{multi}}-A_{\mathrm{single}}}{100-A_{\mathrm{single}}}\,(1+\tfrac{A_{\mathrm{single}}}{100})^2\times100\%$, which penalizes high single-task baselines quadratically.

What would settle it

Recompute transfer for every method in Table 2 using a pre-existing standard measure, such as the plain average gain $\bar{A}_{\mathrm{multi}}-\bar{A}_{\mathrm{single}}$ or the mean per-task relative improvement. If under a standard metric PHP no longer shows positive transfer while all baselines stay negative, the paper's distinctive claim fails; if several baselines also turn positive, the claim that PHP is the only transferring method fails.

Watch

Extended reading notes

Core claim

The paper's claim is that catastrophic forgetting and positive transfer are not opposing forces but consequences of where in the network knowledge is stored. PHP places a task-shared modality aggregating (TMA) adapter at shallow layers to build universal audio-visual correspondences, a task-specific modality-shared dynamic generating (TMDG) adapter at middle layers that synthesizes instance-aware prompts from a prompt pool, and task-specific modality-independent (TMI) prompts at deep layers that preserve each task's and each modality's fine details. Because shallow knowledge is shared, later tasks can benefit from earlier ones, while deep task-specific prompts protect old-task details from being overwritten. The experiments report state-of-the-art accuracy and the lowest forgetting among fine-tuning, EWC, L2P, S-prompt, DualPrompt, PC, and DCNet, together with the only positive transfer score, $+7.79\%$.

Load-bearing premise

The headline transfer result rests on the paper's own Diff metric, whose quadratic penalty on already-strong single-task baselines is a design choice; if that metric is not accepted, the $+7.79\%$ positive-transfer claim has no standard external benchmark behind it.

Editorial extensions

If this is right

  • A single frozen backbone pair (CLIP plus CLAP) can serve four different audio-visual tasks presented sequentially, with accuracy on later tasks meeting or exceeding single-task training on some tasks.
  • The shallow shared adapter carries most of the transfer: in the ablations, TMA alone yields $+8.05\%$ Diff, and the full S-M-D ordering beats every alternative ordering of the three components on both forgetting and transfer.
  • Prompt length and the depth of task-specific layers are tunable levers with opposing effects: longer prompts resist forgetting, while excessive task-specific parameterization degrades cross-task transfer.
  • The method extends to four-task sequences in different orders, holding first-task accuracy stable while absorbing three later tasks.
  • Across all compared baselines, PHP is the only method whose multi-task average exceeds its single-task average, which is the paper's evidence that incremental training can be a positive rather than negative influence.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the shared-to-specific depth ordering is the real cause of positive transfer, the same S-M-D recipe (shared adapters shallow, task-specific prompts deep) is worth testing on other multi-modal continual-learning streams, such as video-language or audio-language task sequences.
  • The Diff metric is a substantive proposal for judging transfer when baselines differ; applying it retroactively to published continual-learning results would reveal whether it changes the ranking of well-known methods, a check the paper does not perform.
  • Because prompt selection here assumes the task index is known at inference time, a natural extension is to let the prompt pool infer task identity from the instance itself, which would make the method usable in task-agnostic streams.
  • The ablation showing TMA alone delivers $+8.05\%$ transfer suggests a minimal PHP variant (shared shallow adapter plus deep TMI prompts, without the middle adapter) might retain most of the benefit at lower parameter cost, a testable simplification.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a three-stage Progressive Homeostatic and Plastic (PHP) prompt tuning method for audio-visual multi-task incremental learning, consisting of a task-shared modality aggregating (TMA) adapter in shallow layers, a task-specific modality-shared dynamic generating (TMDG) adapter in middle layers, and task-specific modality-independent (TMI) prompts in deep layers. The authors claim state-of-the-art performance on four audio-visual tasks (AVE, AVVP, AVS, AVQA) in both anti-forgetting and transfer ability, and they introduce a new transfer metric, Diff, to support the claim that PHP is the only method with positive multi-task transfer.

Significance. The targeted problem—continual learning across multiple audio-visual tasks—is timely and relevant, and the paper offers a concrete modular design with a plausible motivation. The authors also provide a large set of experiments, including per-order results in the supplementary material and a promise of code. However, the central design claim that the three-stage architecture achieves the reported balance is not supported by the paper's own ablations, and the transfer claim rests on a bespoke metric. As the primary contributions are thus not established, the paper's current form does not make a convincing case for its stated results.

major comments (4)
  1. [Tables 3 and 4] The full three-stage model is not the best configuration in the paper's own ablations. In Table 3 (anti-forgetting), row 5 (TMA+TMDG) outperforms row 8 (full PHP) on Amean (59.54 vs 58.85), Afinal (56.01 vs 54.74), and Fmean (3.21 vs 3.32). In Table 4 (transfer), row 2 (TMA alone) has Diff +8.05 vs the full model's +7.77, and row 6 (TMA+TMI) has +7.99. Thus on no metric is the full three-stage model the best. This directly contradicts the abstract and contribution list, which attribute the reported performance to the three-stage progressive design. The design claim is load-bearing and is undermined by the evidence presented.
  2. [Table 7] The four-task incremental learning results, which support the abstract's claim of "SOTA performance in different orders of four tasks," are presented without any baseline comparison. Table 7 shows only the proposed method's per-stage accuracies for two four-task sequences. Without comparing to existing incremental learning methods or even to the ablated variants in Tables 3 and 4, these numbers cannot validate a state-of-the-art claim. This is a major evidential gap for a central assertion of the paper.
  3. [Supplementary Eq. (19)] The transfer metric Diff is introduced in this paper and is not a standard measure. The headline result that PHP is the only method with positive transfer (+7.79%) depends on this metric's quadratic baseline penalty, which the authors choose without standard justification. The paper does not compare Diff with common transfer metrics (e.g., simple relative change or per-task differences), nor does it analyze how the ranking of methods changes under alternative normalizations. Since the claim of positive transfer is a unique selling point of the paper, the metric's form is load-bearing, and its current ad hoc derivation is insufficient.
  4. [Table 1 and Section 4.1] The text states that "our approach outperforms the other methods in terms of all metrics," but Table 1 shows per-task cases where baselines are better: for AVE, Dualprompt has higher Amean (63.00 vs 62.03) and S-prompt has lower Fmean (4.78 vs 6.72); for AVQA, PC has higher Amean (69.55 vs 69.29) and Afinal (69.46 vs 68.56). Only the overall averaged metrics favor PHP. The wording is therefore inaccurate and overstates the comparison, requiring correction or qualification.
minor comments (6)
  1. [Conclusion] The conclusion says "Extensive experiments on three audio-visual tasks (AVE, AVVP, AVS and AVQA)", but four tasks are listed; this should be corrected.
  2. [Section 3.5, Eq. (14)] Equation (14) defines P Xa = concat(Pa, V), but this should likely be concat(Pa, A) to match the audio branch; as written it uses the visual feature V.
  3. [Section 4.2] The heading "Ablation study on deep prompts" appears twice consecutively in the text.
  4. [References] The reference list contains duplicates: [35] and [36] are the same paper, [74] and [75] are the same, and [80] and [81] are the same.
  5. [Supplementary Eq. (19)-(20)] The Diff definition in Eq. (19) and the epsilon-corrected version in Eq. (20) are not reconciled; the main text refers only to Eq. (19), leaving ambiguity about which formula generated Table 2.
  6. [Section 4.3] The heading "Quantitative Results" introduces a section that is entirely qualitative (Figure 5 and descriptive comparisons); the heading should be changed to "Qualitative Results".

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the transfer metric is self-defined but fully disclosed and not fitted to force the result, and no load-bearing argument reduces to a self-citation.

full rationale

The paper's central mechanism is a three-stage prompt adapter design whose components are evaluated against external baselines (CLIP, CLAP, L2P, DualPrompt, PC, etc.) and against each other in ablations. The design motivation cites external works ([13], [41], [82]) rather than the authors' own prior results, and the self-citations present in the reference list are not load-bearing in the derivation. The only bespoke element is the normalized penalty-aware difference metric (Diff, supplementary Eq. 19), which the paper uses to claim that PHP is the only method with positive transfer (+7.79%). While this metric is introduced by the authors, it is a fully specified evaluation rule rather than a fitted parameter: the sign of Diff is determined by the measured sign of Amulti minus Asingle, the penalty term only scales the magnitude, and the metric does not encode any property of PHP that would force it to win. No equation in the paper is equivalent to its own input by construction, and no prediction is renamed from a fitted value. The paper's own ablations do show that the full three-stage model is not the best on every internal metric (e.g., TMA+TMDG has lower Fmean in Table 3, and TMA alone has higher Diff in Table 4), which weakens the causal attribution of the three-stage design, but that is a correctness or internal-consistency concern, not circularity. Therefore the appropriate finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

No fundamentally new entities are postulated; the modules are architectural components. The main ledger-relevant items are hyperparameters (prompt length, pool size, layer depth) and the custom Diff metric penalty, all chosen by hand on a single task sequence.

free parameters (4)
  • Prompt length n = not stated globally; tuned in Fig 3(a) on AVE to AVQA to AVVP sequence
    The length of generated prompts affects the forgetting-transfer trade-off and is selected per sequence.
  • Prompt pool size L = not reported
    Size of the prompt pool in TMDG; no sensitivity analysis or reported value.
  • Number of task-specific layers (TMDG/TMI depth) = not reported; Fig 3(b) sweeps layer count
    Depth of middle and deep stages is chosen to balance AVE and AVVP performance on one sequence.
  • Diff metric penalty exponent = 2 (quadratic)
    The exponent 2 in the baseline penalty (1 + Asingle/100)^2 is chosen ad hoc in supplementary Eq. 19 to penalize high baselines.
assumptions (3)
  • domain assumption Frozen CLIP and CLAP features are sufficient backbones for all four audio-visual tasks.
    Section B supplementary freezes both backbones and only tunes prompts/adapters; if these features lack task-critical information, the method's premise fails.
  • domain assumption Shallow layers encode general structure and deep layers encode task-specific discriminative details, so the S-M-D ordering is the correct inductive bias.
    Section 3.3 cites [82] and [41]; this hierarchy is assumed, but the paper's own ablations (Tables 5-6) show alternative orders perform much worse, yet no mechanism explains why.
  • domain assumption The three attention mechanisms (channel, spatial, temporal) from DG-SCT improve audio-visual fusion in the incremental setting.
    Section 3.3 draws on [13] without re-derivation; the fused representations are assumed beneficial, but ablations show TMA alone can outperform the full model.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Progressive Homeostatic and Plastic Prompt Tuning for Audio-Visual Multi-Task Incremental Learning." pith.science (2026). https://pith.science/paper/FICZBTQH

@misc{pith2026250721588,
  author       = {Pith},
  title        = {Pith review of: Progressive Homeostatic and Plastic Prompt Tuning for Audio-Visual Multi-Task Incremental Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FICZBTQH}},
  note         = {Machine review of arXiv:2507.21588}
}
read the original abstract

Audio-visual multi-task incremental learning aims to continuously learn from multiple audio-visual tasks without the need for joint training on all tasks. The challenge of the problem is how to preserve the old task knowledge while facilitating the learning of new task with previous experiences. To address these challenges, we introduce a three-stage Progressive Homeostatic and Plastic audio-visual prompt (PHP) method. In the shallow phase, we design the task-shared modality aggregating adapter to foster cross-task and cross-modal audio-visual representation learning to enhance shared understanding between tasks. In the middle phase, we propose the task-specific modality-shared dynamic generating adapter, which constructs prompts that are tailored to individual tasks while remaining general across modalities, which balances the models ability to retain knowledge against forgetting with its potential for versatile multi-task transferability. In the deep phase, we introduce the task-specific modality-independent prompts to further refine the understand ability by targeting individual information for each task and modality. By incorporating these three phases, PHP retains task-specific prompts while adapting shared parameters for new tasks to effectively balance knowledge sharing and specificity. Our method achieves SOTA performance in different orders of four tasks (AVE, AVVP, AVS and AVQA). Our code can be available at https://github.com/ENJOY-Yin-jiong/PHP.

Figures

Figures reproduced from arXiv: 2507.21588 by the authors.

Figure 1
Figure 1. (a) Audio-visual multi-task learning allows simulta￾neous training on various audio and video tasks. But, the model may require retraining when a new task emerges. (b) Audio-visual multi-task incremental learning is able to address new tasks more flexibly while maintaining the memory of previous tasks. (c) The pipeline of our method. trained uniformly is deployed, it often struggles to update continuously to acquire… view at source ↗
Figure 2
Figure 2. The Progressive Homeostatic and Plastic audio-visual prompt (PHP) framework consists of three stages for balancing knowledge [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Ablation studies on key design parameters. (a) Impact of [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Performance analysis of multi-task training sequences. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Qualitative Results. We compare our method with base [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

86 extracted references · 67 canonical work pages

  1. [1]

    Memory aware synapses: Learning what (not) to forget

    Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European conference on computer vision (ECCV), pages 139–154, 2018. 1, 2

    2018

  2. [2]

    Visual prompting: Modifying pixel space to adapt pre-trained models

    Hyojin Bahng, Ali Jahanian, Swami Sankaranarayanan, and Phillip Isola. Visual prompting: Modifying pixel space to adapt pre-trained models. arXiv preprint arXiv:2203.17274, 3(11-12):3, 2022. 3

    arXiv 2022

  3. [3]

    Dark experience for gen- eral continual learning: a strong, simple baseline

    Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for gen- eral continual learning: a strong, simple baseline. Advances in neural information processing systems, 33:15920–15930,

  4. [4]

    Co2l: Con- trastive continual learning

    Hyuntak Cha, Jaeho Lee, and Jinwoo Shin. Co2l: Con- trastive continual learning. In Proceedings of the IEEE/CVF International conference on computer vision , pages 9516– 9525, 2021

    2021

  5. [5]

    On tiny episodic memories in continual learning

    Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K Dokania, Philip HS Torr, and Marcaurelio Ranzato. On tiny episodic memories in continual learning. arxiv. Learning, 6(7), 2019. 3

    2019

  6. [6]

    PromptFusion: Decoupling Stability and Plasticity for Continual Learning

    Haoran Chen, Zuxuan Wu, Xintong Han, Menglin Jia, and Yu-Gang Jiang. Promptfusion: Decoupling stabil- ity and plasticity for continual learning. arXiv preprint arXiv:2303.07223, 2023. 1, 3

    work page Pith review arXiv 2023

  7. [7]

    Hts-at: A hierarchi- cal token-semantic audio transformer for sound classifica- tion and detection

    Ke Chen, Xingjian Du, Bilei Zhu, Zejun Ma, Taylor Berg- Kirkpatrick, and Shlomo Dubnov. Hts-at: A hierarchi- cal token-semantic audio transformer for sound classifica- tion and detection. In ICASSP 2022-2022 IEEE Interna- tional Conference on Acoustics, Speech and Signal Process- ing (ICASSP), pages 646–650. IEEE, 2022. 1

    2022

  8. [8]

    Stochas- tic context consistency reasoning for domain adaptive object detection

    Yiming Cui, Liang Li, Jiehua Zhang, Chenggang Yan, Hongkui Wang, Shuai Wang, Heng Jin, and Li Wu. Stochas- tic context consistency reasoning for domain adaptive object detection. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 1331–1340, 2024. 2

    2024

Show all 86 references
  1. [9]

    Prompt customization for con- tinual learning

    Yong Dai, Xiaopeng Hong, Yabin Wang, Zhiheng Ma, Dong- mei Jiang, and Yaowei Wang. Prompt customization for con- tinual learning. arXiv preprint arXiv:2404.18060, 2024. 2, 3, 5

    2024 arXiv

  2. [10]

    Multi-granularity class prototype topology distillation for class-incremental source- free unsupervised domain adaptation

    Peihua Deng, Jiehua Zhang, Xichun Sheng, Chenggang Yan, Yaoqi Sun, Ying Fu, and Liang Li. Multi-granularity class prototype topology distillation for class-incremental source- free unsupervised domain adaptation. In Proceedings of the Computer Vision and Pattern Recognition Co...

    2025

  3. [11]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition ...

    2021

  4. [12]

    Dytox: Transformers for continual learning with dynamic token expansion

    Arthur Douillard, Alexandre Ram ´e, Guillaume Couairon, and Matthieu Cord. Dytox: Transformers for continual learning with dynamic token expansion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9285–9295, 2022. 3

    2022

  5. [13]

    Cross-modal prompts: Adapting large pre- trained models for audio-visual downstream tasks

    Haoyi Duan, Yan Xia, Zhou Mingze, Li Tang, Jieming Zhu, and Zhou Zhao. Cross-modal prompts: Adapting large pre- trained models for audio-visual downstream tasks. Advances in Neural Information Processing Systems, 36, 2024. 3

    2024

  6. [14]

    Adversarial continual learn- ing

    Sayna Ebrahimi, Franziska Meier, Roberto Calandra, Trevor Darrell, and Marcus Rohrbach. Adversarial continual learn- ing. In Computer Vision–ECCV 2020: 16th European Con- ference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI 16, pages 386–402. Springer, 2020. 1, 3

    2020

  7. [15]

    Clap learning audio concepts from nat- ural language supervision

    Benjamin Elizalde, Soham Deshmukh, Mahmoud Al Ismail, and Huaming Wang. Clap learning audio concepts from nat- ural language supervision. In ICASSP 2023-2023 IEEE In- ternational Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), pages 1–5. IEEE, 2023. 3

    2023

  8. [16]

    Revisit weakly-supervised audio-visual video parsing from the lan- guage perspective

    Yingying Fan, Yu Wu, Bo Du, and Yutian Lin. Revisit weakly-supervised audio-visual video parsing from the lan- guage perspective. Advances in Neural Information Process- ing Systems, 36, 2024. 2

    2024

  9. [17]

    Avsegformer: Audio-visual segmentation with trans- former

    Shengyi Gao, Zhe Chen, Guo Chen, Wenhai Wang, and Tong Lu. Avsegformer: Audio-visual segmentation with trans- former. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 12155–12163, 2024. 2

    2024

  10. [18]

    Preface: Brain-inspired ai research

    YiHong Gong and GuoYin Wang. Preface: Brain-inspired ai research. Science China Technological Sciences , 67(8): 2281–2281, 2024. 3

    2024

  11. [19]

    Prompt learning with knowl- edge regularization for pre-trained vision-language models

    Boyang Guo, Liang Li, Jiehua Zhang, Yaoqi Sun, Cheng- gang Yan, and Xichun Sheng. Prompt learning with knowl- edge regularization for pre-trained vision-language models. IEEE Transactions on Multimedia, 2025. Accepted for pub- lication. 2

    2025

  12. [20]

    Curiosity-driven class-incremental learning via adaptive sample selection

    Qinghua Hu, Yucong Gao, and Bing Cao. Curiosity-driven class-incremental learning via adaptive sample selection. IEEE Transactions on Circuits and Systems for Video Tech- nology, 32(12):8660–8673, 2022. 3

    2022

  13. [21]

    Unsupervised prompt learning for vision-language models

    Tony Huang, Jack Chu, and Fangyun Wei. Unsupervised prompt learning for vision-language models. arXiv preprint arXiv:2204.03649, 2022. 2, 3

    2022 arXiv

  14. [22]

    Generating instance-level prompts for rehearsal-free continual learning

    Dahuin Jung, Dongyoon Han, Jihwan Bang, and Hwanjun Song. Generating instance-level prompts for rehearsal-free continual learning. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 11847–11857,

  15. [23]

    Contin- ual learning of a mixed sequence of similar and dissimilar tasks

    Zixuan Ke, Bing Liu, and Xingchang Huang. Contin- ual learning of a mixed sequence of similar and dissimilar tasks. Advances in neural information processing systems , 33:18493–18504, 2020. 2

    2020

  16. [24]

    Overcoming catastrophic forgetting in neu- ral networks

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barwinska, et al. Overcoming catastrophic forgetting in neu- ral networks. Proceedings of the national academy of sc...

    2017

  17. [25]

    Coca: Collaborative causal regulariza- tion for audio-visual question answering

    Mingrui Lao, Nan Pu, Yu Liu, Kai He, Erwin M Bakker, and Michael S Lew. Coca: Collaborative causal regulariza- tion for audio-visual question answering. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 12995– 13003, 2023. 2

    2023

  18. [26]

    Learning to answer questions in dynamic audio-visual scenarios

    Guangyao Li, Yake Wei, Yapeng Tian, Chenliang Xu, Ji- Rong Wen, and Di Hu. Learning to answer questions in dynamic audio-visual scenarios. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19108–19118, 2022. 1

    2022

  19. [27]

    Progressive spatio- temporal perception for audio-visual question answering

    Guangyao Li, Wenxuan Hou, and Di Hu. Progressive spatio- temporal perception for audio-visual question answering. In Proceedings of the 31st ACM International Conference on Multimedia, pages 7808–7816, 2023. 2

    2023

  20. [28]

    Learn to grow: A continual structure learn- ing framework for overcoming catastrophic forgetting

    Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher, and Caiming Xiong. Learn to grow: A continual structure learn- ing framework for overcoming catastrophic forgetting. In International conference on machine learning, pages 3925–

  21. [29]

    Learning without forgetting

    Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelli- gence, 40(12):2935–2947, 2017. 1, 2

    2017

  22. [30]

    Flowgananomaly: Flow-based anomaly network intrusion detection with adver- sarial learning

    Zeyi Li, Pan Wang, and Zixuan Wang. Flowgananomaly: Flow-based anomaly network intrusion detection with adver- sarial learning. Chinese Journal of Electronics, 33(1):58–71,

  23. [31]

    Dual- modality seq2seq network for audio-visual event localiza- tion

    Yan-Bo Lin, Yu-Jhe Li, and Yu-Chiang Frank Wang. Dual- modality seq2seq network for audio-visual event localiza- tion. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2002–2006. IEEE, 2019. 2

    2019

  24. [32]

    Exploring cross-video and cross- modality signals for weakly-supervised audio-visual video parsing

    Yan-Bo Lin, Hung-Yu Tseng, Hsin-Ying Lee, Yen-Yu Lin, and Ming-Hsuan Yang. Exploring cross-video and cross- modality signals for weakly-supervised audio-visual video parsing. Advances in Neural Information Processing Sys- tems, 34:11449–11461, 2021. 2

    2021

  25. [33]

    Annotation-free audio-visual segmentation

    Jinxiang Liu, Yu Wang, Chen Ju, Chaofan Ma, Ya Zhang, and Weidi Xie. Annotation-free audio-visual segmentation. In Proceedings of the IEEE/CVF Winter Conference on Ap- plications of Computer Vision, pages 5604–5614, 2024. 2

    2024

  26. [34]

    Entity-enhanced adaptive reconstruction network for weakly supervised re- ferring expression grounding

    Xuejing Liu, Liang Li, Shuhui Wang, Zheng-Jun Zha, Zechao Li, Qi Tian, and Qingming Huang. Entity-enhanced adaptive reconstruction network for weakly supervised re- ferring expression grounding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3):3003–3018, 2022. 2

    2022

  27. [36]

    A com- parison of statistical learning of naturalistic textures between dcnns and the human visual hierarchy

    XinCheng Lu, ZiQi Yuan, YiChi Zhang, HaiLin Ai, SiYuan Cheng, YiRan Ge, Fang Fang, and NiHong Chen. A com- parison of statistical learning of naturalistic textures between dcnns and the human visual hierarchy. Science China Tech- nological Sciences, 67(8):2310–2318, 2024. 3

    2024

  28. [37]

    Look, lis- ten, and answer: Overcoming biases for audio-visual ques- tion answering

    Jie Ma, Min Hu, Pinghui Wang, Wangchun Sun, Lingyun Song, Hongbin Pei, Jun Liu, and Youtian Du. Look, lis- ten, and answer: Overcoming biases for audio-visual ques- tion answering. arXiv preprint arXiv:2404.12020, 2024. 2

    2024 arXiv

  29. [38]

    Packnet: Adding mul- tiple tasks to a single network by iterative pruning

    Arun Mallya and Svetlana Lazebnik. Packnet: Adding mul- tiple tasks to a single network by iterative pruning. In Pro- ceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 7765–7773, 2018. 2

    2018

  30. [39]

    Catastrophic inter- ference in connectionist networks: The sequential learning problem

    Michael McCloskey and Neal J Cohen. Catastrophic inter- ference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation , pages 109–165. Elsevier, 1989. 2

    1989

  31. [40]

    Multi-modal grouping network for weakly-supervised audio-visual video parsing

    Shentong Mo and Yapeng Tian. Multi-modal grouping network for weakly-supervised audio-visual video parsing. Advances in Neural Information Processing Systems , 35: 34722–34733, 2022. 2

    2022

  32. [41]

    How do llms acquire new knowledge? a knowledge circuits perspective on continual pre-training

    Yixin Ou, Yunzhi Yao, Ningyu Zhang, Hui Jin, Jiacheng Sun, Shumin Deng, Zhenguo Li, and Huajun Chen. How do llms acquire new knowledge? a knowledge circuits perspective on continual pre-training. arXiv preprint arXiv:2502.11196, 2025. 2

    2025 arXiv

  33. [42]

    Fast cross-platform binary code similarity detection frame- work based on cfgs taking advantage of nlp and inductive gnn

    Jinxue Peng, Yong Wang, Jingfeng Xue, and Zhenyan Liu. Fast cross-platform binary code similarity detection frame- work based on cfgs taking advantage of nlp and inductive gnn. Chinese Journal of Electronics, 33(1):128–138, 2024. 2

    2024

  34. [43]

    Dualnet: Con- tinual learning, fast and slow

    Quang Pham, Chenghao Liu, and Steven Hoi. Dualnet: Con- tinual learning, fast and slow. Advances in Neural Informa- tion Processing Systems, 34:16131–16144, 2021. 1, 3

    2021

  35. [44]

    Gdumb: A simple approach that questions our progress in continual learning

    Ameya Prabhu, Philip HS Torr, and Puneet K Dokania. Gdumb: A simple approach that questions our progress in continual learning. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16, pages 524–540. Springer, 2020. 1, 3

    2020

  36. [45]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  37. [46]

    Random path selec- tion for continual learning

    Jathushan Rajasegaran, Munawar Hayat, Salman H Khan, Fahad Shahbaz Khan, and Ling Shao. Random path selec- tion for continual learning. Advances in neural information processing systems, 32, 2019. 2

    2019

  38. [47]

    Progressive neural networks

    Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Raz- van Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv:1606.04671, 2016. 2

    2016 arXiv

  39. [48]

    Overcoming catastrophic forgetting with hard attention to the task

    Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In International conference on machine learning, pages 4548–4557. PMLR, 2018. 2

    2018

  40. [49]

    Incrementer: Transformer for class-incremental semantic segmentation with knowl- edge distillation focusing on old class

    Chao Shang, Hongliang Li, Fanman Meng, Qingbo Wu, Heqian Qiu, and Lanxiao Wang. Incrementer: Transformer for class-incremental semantic segmentation with knowl- edge distillation focusing on old class. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Re...

    2023

  41. [50]

    Privacy-preserving deep learning

    Reza Shokri and Vitaly Shmatikov. Privacy-preserving deep learning. In Proceedings of the 22nd ACM SIGSAC con- ference on computer and communications security , pages 1310–1321, 2015. 3

    2015

  42. [51]

    Coda-prompt: Contin- ual decomposed attention-based prompting for rehearsal-free continual learning

    James Seale Smith, Leonid Karlinsky, Vyshnavi Gutta, Paola Cascante-Bonilla, Donghyun Kim, Assaf Arbelle, Rameswar Panda, Rogerio Feris, and Zsolt Kira. Coda-prompt: Contin- ual decomposed attention-based prompting for rehearsal-free continual learning. In Proceedings of the I...

    2023

  43. [52]

    Audio-visual event localization in unconstrained videos

    Yapeng Tian, Jing Shi, Bochen Li, Zhiyao Duan, and Chen- liang Xu. Audio-visual event localization in unconstrained videos. In Proceedings of the European conference on com- puter vision (ECCV), pages 247–263, 2018. 2, 1

    2018

  44. [53]

    Audio-visual event localization in the wild

    Yapeng Tian, Jing Shi, Bochen Li, Zhiyao Duan, and Chen- liang Xu. Audio-visual event localization in the wild. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition workshops, 2019. 2

    2019

  45. [54]

    Unified mul- tisensory perception: Weakly-supervised audio-visual video parsing

    Yapeng Tian, Dingzeyu Li, and Chenliang Xu. Unified mul- tisensory perception: Weakly-supervised audio-visual video parsing. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceed- ings, Part III 16, pages 436–454. Springer, 2020. 2, 1

    2020

  46. [55]

    Transformer-based under- sampled single-pixel imaging

    Ye Tian, Ying Fu, and Jun Zhang. Transformer-based under- sampled single-pixel imaging. Chinese Journal of Electron- ics, 32(5):1151–1159, 2023. 2

    2023

  47. [56]

    Smart: Syntax-calibrated multi-aspect relation trans- former for change captioning

    Yunbin Tu, Liang Li, Li Su, Zheng-Jun Zha, and Qingming Huang. Smart: Syntax-calibrated multi-aspect relation trans- former for change captioning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(7):4926–4943, 2024. 2

    2024

  48. [57]

    Brain-inspired artificial intelligence research: A review

    GuoYin Wang, HuaNan Bao, Qun Liu, TianGang Zhou, Si Wu, TieJun Huang, ZhaoFei Yu, CeWu Lu, YiHong Gong, ZhaoXiang Zhang, et al. Brain-inspired artificial intelligence research: A review. Science China Technological Sciences, 67(8):2282–2296, 2024. 3

    2024

  49. [58]

    Semantic and relation modulation for audio- visual event localization

    Hao Wang, Zheng-Jun Zha, Liang Li, Xuejin Chen, and Jiebo Luo. Semantic and relation modulation for audio- visual event localization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(6):7711–7725, 2022. 2

    2022

  50. [59]

    On the discrimination and consistency for exemplar-free class incre- mental learning

    Tianqi Wang, Jingcai Guo, Depeng Li, and Zhi Chen. On the discrimination and consistency for exemplar-free class incre- mental learning. arXiv preprint arXiv:2501.15454, 2025. 5

    2025 arXiv

  51. [60]

    S-prompts learning with pre-trained transformers: An occam’s razor for domain incremental learning

    Yabin Wang, Zhiwu Huang, and Xiaopeng Hong. S-prompts learning with pre-trained transformers: An occam’s razor for domain incremental learning. Advances in Neural Informa- tion Processing Systems, 35:5682–5695, 2022. 2, 3, 5

    2022

  52. [61]

    Dualprompt: Complementary prompting for rehearsal-free continual learning

    Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen-Yu Lee, Xiaoqi Ren, Guolong Su, Vin- cent Perot, Jennifer Dy, et al. Dualprompt: Complementary prompting for rehearsal-free continual learning. InEuropean Conference on Computer Vision , pages 631–648. Springer,

  53. [62]

    Learning to prompt for con- tinual learning

    Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jen- nifer Dy, and Tomas Pfister. Learning to prompt for con- tinual learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 139–149,

  54. [63]

    Exploring heterogeneous clues for weakly-supervised audio-visual video parsing

    Yu Wu and Yi Yang. Exploring heterogeneous clues for weakly-supervised audio-visual video parsing. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1326–1335, 2021. 2

    2021

  55. [64]

    Large scale incre- mental learning

    Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incre- mental learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 374–382,

  56. [65]

    Dual attention matching for audio-visual event localization

    Yu Wu, Linchao Zhu, Yan Yan, and Yi Yang. Dual attention matching for audio-visual event localization. In Proceedings of the IEEE/CVF international conference on computer vi- sion, pages 6292–6300, 2019. 2

    2019

  57. [66]

    Deep multi-view enhancement hashing for image retrieval

    Chenggang Yan, Biao Gong, Yuxuan Wei, and Yue Gao. Deep multi-view enhancement hashing for image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 43(4):1445–1451, 2020. 2

    2020

  58. [67]

    Depth image denois- ing using nuclear norm and learning graph model

    Chenggang Yan, Zhisheng Li, Yongbing Zhang, Yutao Liu, Xiangyang Ji, and Yongdong Zhang. Depth image denois- ing using nuclear norm and learning graph model. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 16(4):1–17, 2020. 2

    2020

  59. [68]

    Task- adaptive attention for image captioning

    Chenggang Yan, Yiming Hao, Liang Li, Jian Yin, Anan Liu, Zhendong Mao, Zhenyu Chen, and Xingyu Gao. Task- adaptive attention for image captioning. IEEE Transactions on Circuits and Systems for Video technology, 32(1):43–51,

  60. [69]

    Precise no-reference image quality evaluation based on distortion identification

    Chenggang Yan, Tong Teng, Yutao Liu, Yongbing Zhang, Haoqian Wang, and Xiangyang Ji. Precise no-reference image quality evaluation based on distortion identification. ACM Transactions on Multimedia Computing, Communica- tions, and Applications (TOMM), 17(3s):1–21, 2021. 2

    2021

  61. [70]

    Age-invariant face recognition by multi-feature fu- sionand decomposition with self-attention

    Chenggang Yan, Lixuan Meng, Liang Li, Jiehua Zhang, Zhan Wang, Jian Yin, Jiyong Zhang, Yaoqi Sun, and Bolun Zheng. Age-invariant face recognition by multi-feature fu- sionand decomposition with self-attention. ACM Transac- tions on Multimedia Computing, Communications, and Ap-...

    2022

  62. [71]

    Review of omni- media content quality evaluation

    Chenggang Yan, Yaoqi Sun, Hao Zhong, Chenwei Zhu, Zun- jie Zhu, Bolun Zheng, and Xiaofei Zhou. Review of omni- media content quality evaluation. J. Signal Process., 38(6): 1111–1143, 2022. 2

    2022

  63. [72]

    Der: Dy- namically expandable representation for class incremental learning

    Shipeng Yan, Jiangwei Xie, and Xuming He. Der: Dy- namically expandable representation for class incremental learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3014–3023,

  64. [73]

    Domain shared and specific prompt learning for incremental monocular depth estimation

    Zhiwen Yang, Liang Li, Jiehua Zhang, Tingyu Wang, Yaoqi Sun, and Chenggang Yan. Domain shared and specific prompt learning for incremental monocular depth estimation. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 8306–8315, 2024. 3

    2024

  65. [75]

    Unsupervised cross-media hashing learning via knowledge graph

    Zhaoda Ye, Xiangteng He, and Yuxin Peng. Unsupervised cross-media hashing learning via knowledge graph. Chinese Journal of Electronics, 31(6):1081–1091, 2022. 2

    2022

  66. [76]

    Lifelong learning with dynamically expandable net- works

    Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable net- works. arXiv preprint arXiv:1708.01547, 2017. 2

    2017 arXiv

  67. [77]

    Contrastive corre- lation preserving replay for online continual learning

    Da Yu, Mingyi Zhang, Mantian Li, Fusheng Zha, Junge Zhang, Lining Sun, and Kaiqi Huang. Contrastive corre- lation preserving replay for online continual learning. IEEE Transactions on Circuits and Systems for Video Technology, 34(1):124–139, 2023. 3

    2023

  68. [78]

    Contin- ual learning through synaptic intelligence

    Friedemann Zenke, Ben Poole, and Surya Ganguli. Contin- ual learning through synaptic intelligence. In International conference on machine learning, pages 3987–3995. PMLR,

  69. [79]

    Inductive state- relabeling adversarial active learning with heuristic clique rescaling

    Beichen Zhang, Liang Li, Shuhui Wang, Shaofei Cai, Zheng- Jun Zha, Qi Tian, and Qingming Huang. Inductive state- relabeling adversarial active learning with heuristic clique rescaling. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 2024. 2

    2024

  70. [80]

    Deep guided attention network for joint denoising and demosaicing in real image

    Tao Zhang, Ying Fu, and Jun Zhang. Deep guided attention network for joint denoising and demosaicing in real image. Chinese Journal of Electronics, 33(1):303–312, 2024

    2024

  71. [81]

    Deep guided attention network for joint denoising and demosaicing in real image

    Tao Zhang, Ying Fu, and Jun Zhang. Deep guided attention network for joint denoising and demosaicing in real image. Chinese Journal of Electronics, 33(1):303–312, 2024. 2

    2024

  72. [82]

    A survey on concept factorization: From shallow to deep representation learning

    Zhao Zhang, Yan Zhang, Mingliang Xu, Li Zhang, Yi Yang, and Shuicheng Yan. A survey on concept factorization: From shallow to deep representation learning. Information Pro- cessing & Management, 58(3):102534, 2021. 3

    2021

  73. [83]

    From speaker to dubber: movie dubbing with prosody and duration consistency learning

    Zhedong Zhang, Liang Li, Gaoxiang Cong, Haibing Yin, Yuhan Gao, Chenggang Yan, Anton van den Hengel, and Yuankai Qi. From speaker to dubber: movie dubbing with prosody and duration consistency learning. In Proceedings of the 32nd ACM International Conference on Multimedia , pa...

    2024

  74. [84]

    Prosody- enhanced acoustic pre-training and acoustic-disentangled prosody adapting for movie dubbing

    Zhedong Zhang, Liang Li, Chenggang Yan, Chunshan Liu, Anton van den Hengel, and Yuankai Qi. Prosody- enhanced acoustic pre-training and acoustic-disentangled prosody adapting for movie dubbing. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 172...

    2025

  75. [85]

    Heterogeneous prompt-guided entity inferring and distilling for scene-text aware cross-modal retrieval

    Zhiqian Zhao, Liang Li, Jiehua Zhang, Yaoqi Sun, Xichun Sheng, Haibing Yin, and Shaowei Jiang. Heterogeneous prompt-guided entity inferring and distilling for scene-text aware cross-modal retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence , pages 10537–10545,

  76. [86]

    Gait recognition in the wild with dense 3d representations and a benchmark

    Jinkai Zheng, Xinchen Liu, Wu Liu, Lingxiao He, Cheng- gang Yan, and Tao Mei. Gait recognition in the wild with dense 3d representations and a benchmark. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 20228–20237, 2022. 2

    2022

  77. [87]

    Audio–visual segmentation

    Jinxing Zhou, Jianyuan Wang, Jiayi Zhang, Weixuan Sun, Jing Zhang, Stan Birchfield, Dan Guo, Lingpeng Kong, Meng Wang, and Yiran Zhong. Audio–visual segmentation. In European Conference on Computer Vision , pages 386–

  78. [403]

    2, 1 Progressive Homeostatic and Plastic Prompt Tuning for Audio-Visual Multi-Task Incremental Learning Supplementary Material A

    Springer, 2022. 2, 1 Progressive Homeostatic and Plastic Prompt Tuning for Audio-Visual Multi-Task Incremental Learning Supplementary Material A. Tasks and Datasets In this work, we simulate audio-visual multi-task incremen- tal learning by treating multiple audio-visual tasks...

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.