Pith. sign in

REVIEW 4 major objections 5 minor 80 references

Information-Theoretic Complementary Prompts for Improved Continual Text Classification

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read InfoComp learns two complementary prompts to classify a stream of text tasks without replaying old data.

desk verdict A real rehearsal-free prompt-based CTC method with clean ablations, but the headline 3.7-point gain rests on an internally inconsistent comparison row and unchecked baseline numbers. read the letter →

arxiv 2505.20933 v1 pith:SHEILYAY submitted 2025-05-27 cs.CL

Duzhen Zhang , Yong Ren , Chenxing Li , Dong Yu , Tielin Zhang This is my paper · ORCID
classification cs.CL
keywords continualtextclassificationcatastrophicforgettingforwardknowledgetransferprompttuningmutualinformationcomplementarylearningsystemsrehearsal-freeparameter-efficientfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

InfoComp claims that continual text classification is best served by splitting the prompt into two roles: a private prompt per task that absorbs task-specific knowledge, and a shared prompt that accumulates task-invariant knowledge across all tasks. The paper argues that previous prompt-based continual learners either ignore shared knowledge or pay a growing computational cost by concatenating all past prompts. InfoComp adds two information-theoretic objectives that push the private prompt to align with its task classifier and push the shared prompt to keep representations consistent with its earlier state. The reported result is that this design, without any data replay, consistently beats prior methods on the standard five-task benchmark (average accuracy 80.0, up 0.9 over the best baseline) and on the fifteen-task long-sequence benchmark (up to 3.7 points in the 20-sample few-shot setting). A sympathetic reader would care because it suggests a cheap, privacy-friendly route to lifelong text classification that keeps prompt length fixed rather than growing with the number of tasks.

What carries the argument

The load-bearing object is the complementary prompt pair: a private P-Prompt per task and a single shared S-Prompt, both soft prompts fed into the frozen pretrained language model. The information-theoretic machinery that carries the argument is the pair of surrogate mutual-information objectives: the inner-product loss $L_{p\text{-info}}^k = -\theta_{\text{head}}^{k\,T} W_1 \theta_{P_k}^T$ is used as a gradient-friendly proxy for $I(\theta_{P_k}; \theta_{\text{head}}^k \mid X_i^k)$, and a SimSiam-style consistency loss $L_{s\text{-info}}^k = -(W_q V_i^k)^T V_i^{k\prime}$ is used as a positive-only contrastive proxy for $I(V_i^k; V_i^{k\prime})$, with a stop-gradient on the previous S-Prompt representation. These two losses let the same fixed-length prompt architecture push task-specific information into the private prompt and shared information into the shared prompt, which is what prevents forgetting and enables forward transfer.

What would settle it

Re-run the cited baselines and InfoComp on the same four task orders and three 15-task orders with identical prompt lengths, epochs, seeds, validation splits, and test sets, and compare paired differences with the reported $p<0.05$ test; if the average gains over the strongest baselines fall below the significance threshold under those controlled conditions, the claim of consistent state-of-the-art improvement is falsified.

Watch

Extended reading notes

Core claim

The central claim is that explicitly learning two complementary prompt spaces improves continual text classification. For each task $T_k$ the model trains a task-specific P-Prompt $P_k$ and a shared S-Prompt $S$, with cross-entropy loss $L_k$ on the input $[P_k, S, X_i^k]$ while the PLM stays frozen. Two mutual-information losses are added: $L_{p\text{-info}}^k$ maximizes the inner product between the P-Prompt parameters and the task classifier parameters, which the paper reads as increasing $I(\theta_{P_k}; \theta^k_{\text{head}} \mid X_i^k)$ and thereby strengthening task-specific knowledge; $L_{s\text{-info}}^k$ maximizes agreement between representations of the same input under the current and the frozen previous S-Prompt, following the SimSiam objective, which preserves task-invariant knowledge across the sequence. The total objective is $L_{\text{overall}}^k = L_k + \lambda_1 L_{p\text{-info}}^k + \lambda_2 L_{s\text{-info}}^k$. The paper reports consistent gains over the strongest prompt-based baselines on four orderings of the standard five-task CTC benchmark and three orderings of a 15-task benchmark, with differences judged significant at $p < 0.05$.

Load-bearing premise

The load-bearing premise is that the baseline numbers taken from earlier papers are directly comparable to InfoComp, since most baselines in Tables 3 and 4 were not re-run under the same prompt lengths, epochs, seeds, and validation protocols; if those configurations differ, the reported gains could be configuration effects rather than a property of the method.

Editorial extensions

If this is right

  • InfoComp keeps total prompt length constant as the number of tasks grows, because only the current P-Prompt and the shared S-Prompt are used; this removes the linear prompt-list growth and its associated computational cost.
  • Rehearsal-free continual text classification can outperform replay-based methods: InfoComp beats the episodic-memory baselines on the standard benchmark while storing no previous examples.
  • The ablation results imply that both prompt types and both mutual-information losses contribute; removing either prompt or either loss lowers accuracy on the 15-task orders.
  • The gains extend to few-shot regimes: at 20 samples per class the method reports a 3.7-point average improvement over the strongest baseline across three 15-task orders.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the S-Prompt consistency loss is a form of self-distillation through prompt states, so InfoComp may transfer to other continual settings with a frozen encoder, such as continual named entity recognition or continual question answering.
  • Editorial inference: because the method is model-agnostic and only trains prompt parameters, a testable extension is applying it to decoder-only language models or multimodal encoders; the inner-product mutual-information surrogate may need recalibration when the classifier head is replaced by a language-model head.
  • Editorial inference: the claimed gains rest on comparisons to baselines cited from earlier papers, so a fully controlled re-run with identical prompt lengths, epochs, seeds, and validation splits is the natural next check.
  • Editorial inference: a direct probe of whether the S-Prompt truly stores task-invariant knowledge would be to freeze it after training and evaluate zero-shot or few-shot transfer to an unseen task; the paper does not report such a probe.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces InfoComp, a rehearsal-free continual text classification (CTC) method that maintains per-task private prompts (P-Prompt) and a shared prompt (S-Prompt), trained with two auxiliary objectives presented as mutual-information maximization (Eqs. (5) and (7)) and combined in Eq. (8). Experiments on a 5-task standard benchmark and 15-task long-sequence benchmark (Tables 3 and 4) report accuracy gains over ProgPrompt and other baselines, with ablations in Table 5 supporting the contribution of each component. The central empirical claim is that InfoComp outperforms prior state-of-the-art methods without data replay, with average gains of 0.9 points on the 5-task benchmark and up to 3.7 points on the 15-task few-shot setting.

Significance. If the results hold under a controlled comparison, InfoComp would be a simple, parameter-efficient, replay-free alternative to progressive prompt concatenation, and the constant prompt length is a practical advantage over ProgPrompt. The ablation study is a genuine strength: it isolates the P-Prompt, S-Prompt, and each auxiliary loss. However, the information-theoretic framing currently rests on heuristic similarity losses rather than demonstrable mutual-information objectives, and the headline improvement is sensitive to the choice of comparison baseline. The empirical claims are promising but need verification before the paper can be accepted.

major comments (4)
  1. [Table 4] The 'Improve' row is computed with an inconsistent baseline. The per-cell improvements (Order5: 1.1/1.7/1.5; Order6: 3.5/2.3/2.4; Order7: 2.6/3.0/1.8) average to about 2.2, while the reported averages 3.7, 2.7, and 1.9 are obtained by comparing InfoComp's average to the reimplemented ProgPrompt* for 20 shots (57.9 - 54.2) and to the cited ProgPrompt for 200 and 1000 shots (69.6 - 66.9 and 71.2 - 69.3). Some per-cell entries also use the stronger of the cited and reimplemented ProgPrompt values. Please fix the table by choosing one comparison protocol and reporting the average of the per-cell differences; the claim of 'up to 3.7 points' is not supported by the current numbers, although a consistent comparison still leaves a positive average gain of roughly 2.2 points.
  2. [Section 4.2, Eqs. (5)-(7)] The paper asserts that maximizing the inner product in Eq. (5) maximizes I(θPk; θhead|X), and that the SimSiam-style objective in Eq. (7) maximizes I(V_i^k; V_i^{k'}), but no derivation or variational bound is provided. A bilinear similarity term does not generally equal or bound mutual information, and the SimSiam objective is a positive-only alignment loss rather than an information-theoretic estimator. Since the method's name and its two main contributions are explicitly information-theoretic, either provide a formal derivation (e.g., an InfoNCE-style lower bound) or reframe these terms as similarity-based regularizers. As written, the central conceptual claim is unsupported.
  3. [Section 5.3, Tables 3 and 4] The † significance markers are unverifiable as reported. The manuscript states only that a paired t-test at the 0.05 level was used; it does not specify the pairing unit (task order? seed?), the number of runs used in the test, the degrees of freedom, or the observed standard deviations. Since all results are averages of 3 runs and no per-run values are given, the statistical significance claims cannot be checked. Please report per-run results with standard deviations and the exact test protocol.
  4. [Section 5.2, Table 4] Most baseline results are cited from prior papers rather than re-run under the same protocol, and the reimplemented ProgPrompt* deviates from the cited numbers by up to 5.2 points (Order7/20: 57.1 vs 51.9), a gap larger than many of the claimed improvements. This makes it hard to rule out configuration differences (prompt length, epochs, validation splits, seeds) as an alternative explanation for the observed gains. Please either re-run all baselines under the identical protocol used for InfoComp, or provide explicit evidence that the cited numbers are compatible with that protocol.
minor comments (5)
  1. [Section 5.4] The implementation details mention a 'G-Prompt length' of 5, but no G-Prompt is defined anywhere in the paper; this appears to be a typo for the S-Prompt length.
  2. [Eqs. (6)-(7) and Figure 3] The stop-gradient operation, which is central to SimSiam and is shown in Figure 3, does not appear in the equations; please state explicitly whether gradients are blocked on V_i^{k'} in Eq. (7).
  3. [Eq. (5)] The expression θhead^T W1 θP^T requires the prompt parameters and classifier weights to be arranged as vectors; please specify how these parameters are flattened and give the shape of W1.
  4. [Section 6.1.1] The sentence reporting 'an improvement of 1.6, 2.3, 2.0, and 2.3 over ProgPrompt' is internally ambiguous: the second value 2.3 matches the reimplemented ProgPrompt* (80.1 - 77.8), while the cited ProgPrompt row gives 80.1 - 77.7 = 2.4. Please clarify which baseline is being compared.
  5. [General] The paper would be easier to reproduce if the authors released code, seeds, and per-run results for the reimplemented baselines, especially because the comparison is sensitive to baseline configuration.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity found: the prompt losses are explicit regularizers and the benchmark gains are evaluated against external baselines, not constructed from the method's own definitions.

full rationale

InfoComp's central derivation is not circular. Equation (4) defines a standard prompt-tuning objective with an added shared prompt; Equations (5) and (7) are explicit, trainable regularizers (an inner-product alignment term and a SimSiam-style consistency term), not quantities fitted to the reported accuracy numbers. The claims that P-Prompt encodes task-specific knowledge and S-Prompt retains task-invariant knowledge are design/terminology choices: P-Prompt is per-task by construction, and Ls-info explicitly aligns current S-Prompt behavior to the frozen previous S-Prompt. The actual contribution claimed, higher average accuracy on five- and fifteen-task benchmarks, is tested against external baselines in Tables 3 and 4 and does not reduce to any fitted parameter or self-referential definition. The mutual-information framing is asserted rather than derived (inner products and SimSiam are used as surrogates), which is a theoretical rigor concern but not circularity. Background self-citations ([22, 24, 25, 35, 42, 43]) are not load-bearing for the central claim. The internally inconsistent 'Improve' row in Table 4 and the mixed cited/reimplemented baseline protocol are reproducibility and comparison-fairness issues, not circularity.

Assumptions & free parameters 6 free parameters · 4 assumptions · 2 invented entities

The empirical accuracy claim rests mainly on benchmark comparisons. The information-theoretic framing adds two unsupported equivalence assumptions (inner products and SimSiam objectives stand in for mutual information) and two prompt-space constructs. No new physical or mathematical entities are introduced beyond the prompt vectors and learned projection matrices W1, Wq.

free parameters (6)
  • lambda_1 = 0.05
    Balancing coefficient for the P-Prompt information loss in Eq. (8); set by hand without sensitivity study.
  • lambda_2 = 0.1
    Balancing coefficient for the S-Prompt information loss in Eq. (8); set by hand without sensitivity study.
  • P-Prompt length = 35 tokens
    Chosen in Section 5.4; no search or analysis reported.
  • S-Prompt length = 5 tokens
    Chosen in Section 5.4 (written as 'G-Prompt length' in the text); no search or analysis reported.
  • Learning rate = 1e-4
    Chosen for all experiments in Section 5.4; no sensitivity study.
  • Epoch counts per setting = 40 (standard); 300/150/40 for 20/200/1000 samples per class
    Ad hoc choices in Section 5.4; no sensitivity study.
assumptions (4)
  • domain assumption The loss in Eq. (5), a linear-projected inner product between prompt and classifier parameters, monotonically increases the mutual information I(theta_Pk; theta_head_k | X_i^k).
    Stated without proof in Section 4.2; not an established information-theoretic bound.
  • domain assumption The SimSiam objective with stop-gradient in Eq. (7) maximizes I(V_i^k; V_i^k') for prompts from consecutive tasks.
    Claimed by analogy to SimSiam [67]; no MI estimator or bound is provided.
  • domain assumption A single sequentially updated S-Prompt can carry task-invariant knowledge without itself forgetting.
    Core to Section 4.1; only indirect support via the ablation in Table 5.
  • domain assumption Task identity is available at training and inference.
    Explicit in Section 3.1, following ProgPrompt [13].
invented entities (2)
  • P-Prompt (per-task private soft prompt) independent evidence
    purpose: Encodes task-specific knowledge for each task to mitigate catastrophic forgetting.
    Ablation in Table 5 ('InfoComp w/o P-Prompt') drops performance substantially, so the component is empirically supported; the claim that it encodes task-specific knowledge is inferred, not directly measured.
  • S-Prompt (shared soft prompt) independent evidence
    purpose: Encodes task-invariant knowledge shared across all tasks to enable forward knowledge transfer without growing the prompt list.
    Ablation in Table 5 ('InfoComp w/o S-Prompt') shows a performance drop, but the task-invariance property is asserted rather than directly measured.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Information-Theoretic Complementary Prompts for Improved Continual Text Classification." pith.science (2026). https://pith.science/paper/SHEILYAY

@misc{pith2026250520933,
  author       = {Pith},
  title        = {Pith review of: Information-Theoretic Complementary Prompts for Improved Continual Text Classification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SHEILYAY}},
  note         = {Machine review of arXiv:2505.20933}
}
read the original abstract

Continual Text Classification (CTC) aims to continuously classify new text data over time while minimizing catastrophic forgetting of previously acquired knowledge. However, existing methods often focus on task-specific knowledge, overlooking the importance of shared, task-agnostic knowledge. Inspired by the complementary learning systems theory, which posits that humans learn continually through the interaction of two systems -- the hippocampus, responsible for forming distinct representations of specific experiences, and the neocortex, which extracts more general and transferable representations from past experiences -- we introduce Information-Theoretic Complementary Prompts (InfoComp), a novel approach for CTC. InfoComp explicitly learns two distinct prompt spaces: P(rivate)-Prompt and S(hared)-Prompt. These respectively encode task-specific and task-invariant knowledge, enabling models to sequentially learn classification tasks without relying on data replay. To promote more informative prompt learning, InfoComp uses an information-theoretic framework that maximizes mutual information between different parameters (or encoded representations). Within this framework, we design two novel loss functions: (1) to strengthen the accumulation of task-specific knowledge in P-Prompt, effectively mitigating catastrophic forgetting, and (2) to enhance the retention of task-invariant knowledge in S-Prompt, improving forward knowledge transfer. Extensive experiments on diverse CTC benchmarks show that our approach outperforms previous state-of-the-art methods.

Figures

Figures reproduced from arXiv: 2505.20933 by the authors.

Figure 1
Figure 1. InfoComp learns two distinct prompt types: P-Prompt and S-Prompt. The [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. Alleviating the issue of forgetting shared knowledge across tasks by maximizing [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. (a) SimSiam for the positive sample pair. (b) SimSiam for the prompt-augmented [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Comparison of task-wise accuracy for the task sequences Order3 and Order4. [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

80 extracted references · 61 canonical work pages

  1. [1]

    Devlin, M.-W

    J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, in: Pro- ceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolo- gies, Volume 1 (Long and Short Papers), 2019, pp. 4171–4186

    work page 2019

  2. [2]

    de Masson D’Autume, S

    C. de Masson D’Autume, S. Ruder, L. Kong, D. Yogatama, Episodic memory in lifelong language learning, Advances in Neural Information Processing Systems 32 (2019)

    work page 2019

  3. [3]

    Huang, Y

    Y. Huang, Y. Zhang, J. Chen, X. Wang, D. Yang, Continual Learning for Text Classification with Information Disentanglement Based Regu- larization, in: Proceedings of the 2021 Conference of the North Ameri- can Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021, pp. 2736–2746

    work page 2021

  4. [4]

    Robins, Catastrophic forgetting, rehearsal and pseudorehearsal, Con- nection Science 7 (2) (1995) 123–146

    A. Robins, Catastrophic forgetting, rehearsal and pseudorehearsal, Con- nection Science 7 (2) (1995) 123–146

    work page 1995

  5. [5]

    McCloskey, N

    M. McCloskey, N. J. Cohen, Catastrophic interference in connectionist networks: The sequential learning problem, in: Psychology of learning and motivation, Vol. 24, Elsevier, 1989, pp. 109–165

    work page 1989

  6. [6]

    I. J. Goodfellow, M. Mirza, D. Xiao, A. Courville, Y. Bengio, An em- pirical investigation of catastrophic forgetting in gradient-based neural networks, arXiv preprint arXiv:1312.6211 (2013)

    arXiv 2013

  7. [7]

    Y. Zhai, S. Tong, X. Li, M. Cai, Q. Qu, Y. J. Lee, Y. Ma, Investigating the catastrophic forgetting in multimodal large language model fine- tuning, in: Conference on Parsimony and Learning, PMLR, 2024, pp. 202–227

    work page 2024

  8. [8]

    Zhang, Y

    H. Zhang, Y. Wu, D. Li, S. Yang, R. Zhao, Y. Jiang, F. Tan, Balanc- ing speciality and versatility: a coarse to fine framework for supervised 21 fine-tuning large language model, in: Findings of the Association for Computational Linguistics ACL 2024, 2024, pp. 7467–7509

    work page 2024

Show all 80 references
  1. [9]

    X. Liu, K. Ji, Y. Fu, W. Tam, Z. Du, Z. Yang, J. Tang, P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks, in: Proceedings of the 60th Annual Meeting of the Association for Com- putational Linguistics (Volume 2: Short Papers), 2022, pp. 61–68

    2022

  2. [10]

    Lester, R

    B. Lester, R. Al-Rfou, N. Constant, The Power of Scale for Parameter- Efficient Prompt Tuning, in: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 3045– 3059

    2021

  3. [11]

    P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, G. Neubig, Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing, ACM Computing Surveys 55 (9) (2023) 1–35

    2023

  4. [12]

    Z. Wang, Z. Zhang, C.-Y. Lee, H. Zhang, R. Sun, X. Ren, G. Su, V. Perot, J. Dy, T. Pfister, Learning to prompt for continual learn- ing, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 139–149

    2022

  5. [13]

    Razdaibiedina, Y

    A. Razdaibiedina, Y. Mao, R. Hou, M. Khabsa, M. Lewis, A. Almahairi, Progressive Prompts: Continual Learning for Language Models, in: The Eleventh International Conference on Learning Representations, 2023

    2023

  6. [14]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, I. Polosukhin, Attention is all you need, Advances in neural information processing systems 30 (2017)

    2017

  7. [15]

    J. L. McClelland, B. L. McNaughton, R. C. O’Reilly, Why there are complementary learning systems in the hippocampus and neocortex: in- sights from the successes and failures of connectionist models of learning and memory., Psychological review 102 (3) (1995) 419

    1995

  8. [16]

    Kumaran, D

    D. Kumaran, D. Hassabis, J. L. McClelland, What learning systems do intelligent agents need? Complementary learning systems theory updated, Trends in cognitive sciences 20 (7) (2016) 512–534. 22

    2016

  9. [17]

    Arani, F

    E. Arani, F. Sarfraz, B. Zonooz, Learning Fast, Learning Slow: A Gen- eral Continual Learning Method based on Complementary Learning Sys- tem, in: International Conference on Learning Representations, 2021

    2021

  10. [18]

    Q. Pham, C. Liu, S. Hoi, Dualnet: Continual learning, fast and slow, Advances in Neural Information Processing Systems 34 (2021) 16131– 16144

    2021

  11. [19]

    Z. Wang, Z. Zhang, S. Ebrahimi, R. Sun, H. Zhang, C.-Y. Lee, X. Ren, G. Su, V. Perot, J. Dy, et al., Dualprompt: Complementary prompt- ing for rehearsal-free continual learning, in: European Conference on Computer Vision, Springer, 2022, pp. 631–648

    2022

  12. [20]

    J. Wu, T. Yu, R. Wang, Z. Song, R. Zhang, H. Zhao, C. Lu, S. Li, R. Henao, InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understanding, arXiv preprint arXiv:2306.04933 (2023)

    2023 arXiv

  13. [21]

    L. Wang, X. Zhang, H. Su, J. Zhu, A Comprehensive Survey of Contin- ual Learning: Theory, Method and Application, CoRR abs/2302.00487 (2023). arXiv:2302.00487, doi:10.48550/ARXIV.2302.00487. URL https://doi.org/10.48550/arXiv.2302.00487

    2023 doi arXiv

  14. [22]

    Zheng, C

    J. Zheng, C. Shi, X. Cai, Q. Li, D. Zhang, C. Li, D. Yu, Q. Ma, Lifelong Learning of Large Language Model based Agents: A Roadmap, arXiv preprint arXiv:2501.07278 (2025)

    2025

  15. [23]

    J. Dong, D. Zhang, Y. Cong, W. Cong, H. Ding, D. Dai, Federated incremental semantic segmentation, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 3934– 3943

    2023

  16. [24]

    Zhang, Y

    D. Zhang, Y. Yu, C. Li, J. Dong, D. Yu, Federated Incremental Named Entity Recognition, IEEE Transactions on Audio, Speech and Language Processing (2025)

    2025

  17. [25]

    J. Dong, W. Liang, H. Li, D. Zhang, M. Cao, H. Ding, S. H. Khan, F. Shahbaz Khan, How to Continually Adapt Text-to-Image Diffusion Models for Flexible Customization?, Advances in Neural Information Processing Systems 37 (2024) 130057–130083. 23

    2024

  18. [26]

    Rebuffi, A

    S.-A. Rebuffi, A. Kolesnikov, G. Sperl, C. H. Lampert, icarl: Incremental classifier and representation learning, in: Proceedings of the IEEE con- ference on Computer Vision and Pattern Recognition, 2017, pp. 2001– 2010

    2017

  19. [27]

    Chaudhry, M

    A. Chaudhry, M. Ranzato, M. Rohrbach, M. Elhoseiny, Efficient Life- long Learning with A-GEM, in: International Conference on Learning Representations, 2018

    2018

  20. [28]

    H. Cha, J. Lee, J. Shin, Co 2L: Contrastive Continual Learning, in: 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, 2021, pp. 9496–9505

    2021

  21. [29]

    Shokri, V

    R. Shokri, V. Shmatikov, Privacy-Preserving Deep Learning, in: Pro- ceedings of the 22nd ACM SIGSAC Conference on Computer and Com- munications Security, Denver, CO, USA, October 12-16, 2015, 2015, pp. 1310–1321

    2015

  22. [30]

    Kirkpatrick, R

    J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, et al., Overcoming catastrophic forgetting in neural networks, Proceed- ings of the national academy of sciences 114 (13) (2017) 3521–3526

    2017

  23. [31]

    Farajtabar, N

    M. Farajtabar, N. Azizan, A. Mott, A. Li, Orthogonal gradient descent for continual learning, in: International Conference on Artificial Intelli- gence and Statistics, PMLR, 2020, pp. 3762–3773

    2020

  24. [32]

    S. Hou, X. Pan, C. C. Loy, Z. Wang, D. Lin, Learning a unified clas- sifier incrementally via rebalancing, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 831–839

    2019

  25. [33]

    Zhang, W

    D. Zhang, W. Cong, J. Dong, Y. Yu, X. Chen, Y. Zhang, Z. Fang, Continual Named Entity Recognition without Catastrophic Forgetting, in: The 2023 Conference on Empirical Methods in Natural Language Processing, 2023

    2023

  26. [34]

    Z. Li, D. Hoiem, Learning without forgetting, IEEE transactions on pattern analysis and machine intelligence 40 (12) (2017) 2935–2947. 24

    2017

  27. [35]

    Zhang, H

    D. Zhang, H. Li, W. Cong, R. Xu, J. Dong, X. Chen, Task relation distillation and prototypical pseudo label for incremental named entity recognition, in: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, 2023, pp. 3319–3329

    2023

  28. [36]

    A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, R. Hadsell, Progressive neural networks, arXiv preprint arXiv:1606.04671 (2016)

    2016 arXiv

  29. [37]

    J. Yoon, E. Yang, J. Lee, S. J. Hwang, Lifelong Learning with Dynam- ically Expandable Networks, in: International Conference on Learning Representations, 2018

    2018

  30. [38]

    Karimi Mahabadi, J

    R. Karimi Mahabadi, J. Henderson, S. Ruder, Compacter: Efficient low-rank hypercomplex adapter layers, Advances in Neural Information Processing Systems 34 (2021) 1022–1035

    2021

  31. [39]

    X. L. Li, P. Liang, Prefix-Tuning: Optimizing Continuous Prompts for Generation, in: Proceedings of the 59th Annual Meeting of the Asso- ciation for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 20...

    2021

  32. [40]

    Y. Gu, X. Han, Z. Liu, M. Huang, PPT: Pre-trained Prompt Tuning for Few-shot Learning, in: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022, pp. 8410–8423

    2022

  33. [41]

    Z. Wang, R. Panda, L. Karlinsky, R. Feris, H. Sun, Y. Kim, Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learning, in: The Eleventh International Conference on Learning Representations, 2023

    2023

  34. [42]

    Z.-Z. Li, D. Zhang, M.-L. Zhang, J. Zhang, Z. Liu, Y. Yao, H. Xu, J. Zheng, P.-J. Wang, X. Chen, et al., From system 1 to system 2: A sur- vey of reasoning large language models, arXiv preprint arXiv:2502.17419 (2025)

    2025 arXiv

  35. [43]

    Zhang, Y

    D. Zhang, Y. Yu, J. Dong, C. Li, D. Su, C. Chu, D. Yu, MM-LLMs: Recent Advances in MultiModal Large Language Models, in: Findings of the Association for Computational Linguistics ACL 2024, 2024, pp. 12401–12430. 25

    2024

  36. [44]

    X. Zhao, X. Luo, Q. Shi, C. Chen, S. Wang, W. Che, Z. Liu, M. Sun, ChartCoder: Advancing Multimodal Large Language Model for Chart- to-Code Generation, arXiv preprint arXiv:2501.06598 (2025)

    2025 arXiv

  37. [45]

    X. Zhao, X. Liu, H. Yang, X. Luo, F. Zeng, J. Li, Q. Shi, C. Chen, ChartEdit: How Far Are MLLMs From Automating Chart Analy- sis? Evaluating MLLMs’ Capability via Chart Editing, arXiv preprint arXiv:2505.11935 (2025)

    2025 arXiv

  38. [46]

    C. Qin, S. Joty, LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5, in: International Conference on Learning Representations, 2022

    2022

  39. [47]

    Q. Zhu, B. Li, F. Mi, X. Zhu, M. Huang, Continual Prompt Tuning for Dialog State Tracking, in: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, 2022, pp. 1124–1137

    2022

  40. [48]

    Z. Wang, Y. Liu, T. Ji, X. Wang, Y. Wu, C. Jiang, Y. Chao, Z. Han, L. Wang, X. Shao, et al., Rehearsal-free continual language learning via efficient parameter isolation, in: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Lon...

    2023

  41. [49]

    Liang, F

    Z. Liang, F. Wei, Y. Jie, Y. Qian, Z. Hao, B. Han, Prompts Can Play Lottery Tickets Well: Achieving Lifelong Information Extraction via Lottery Prompt Tuning, in: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), A...

    2023

  42. [50]

    Y. Liu, Z. Xiong, Y. Yuan, Q. Wang, Transcending pixels: boosting saliency detection via scene understanding from aerial imagery, IEEE Transactions on Geoscience and Remote Sensing 61 (2023) 1–16

    2023

  43. [51]

    Y. Bai, Y. Liu, Y. Li, Learning frequency-aware cross-modal interaction for multimodal fake news detection, IEEE Transactions on Computa- tional Social Systems (2024)

    2024

  44. [52]

    C. E. Shannon, A mathematical theory of communication, The Bell system technical journal 27 (3) (1948) 379–423. 26

    1948

  45. [53]

    C. E. Shannon, Prediction and entropy of printed English, Bell system technical journal 30 (1) (1951) 50–64

    1951

  46. [54]

    T. M. Cover, Elements of information theory, John Wiley & Sons, 1999

    1999

  47. [55]

    P. F. Brown, V. J. Della Pietra, P. V. Desouza, J. C. Lai, R. L. Mercer, Class-based n-gram models of natural language, Computational linguis- tics 18 (4) (1992) 467–480

    1992

  48. [56]

    Manning, H

    C. Manning, H. Schutze, Foundations of statistical natural language processing, MIT press, 1999

    1999

  49. [57]

    Tishby, F

    N. Tishby, F. C. Pereira, W. Bialek, The information bottleneck method, arXiv preprint physics/0004057 (2000)

    2000 arXiv

  50. [58]

    Steinborn, P

    V. Steinborn, P. Dufter, H. Jabbar, H. Schuetze, An information- theoretic approach and dataset for probing gender stereotypes in mul- tilingual masked language models, in: Findings of the Association for Computational Linguistics: NAACL 2022, 2022, pp. 921–932

    2022

  51. [59]

    P. West, A. Holtzman, J. Buys, Y. Choi, Bottlesum: Unsupervised and self-supervised sentence summarization using the information bottleneck principle, in: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Con...

    2019

  52. [60]

    J. Ju, M. Liu, H. Y. Koh, Y. Jin, L. Du, S. Pan, Leveraging informa- tion bottleneck for scientific document summarization, in: Empirical Methods in Natural Language Processing 2021, Association for Compu- tational Linguistics (ACL), 2021, pp. 4091–4098

    2021

  53. [61]

    B. Ji, T. Zhang, Y. Zou, B. Hu, S. Shen, Increasing Visual Awareness in Multimodal Neural Machine Translation from an Information Theo- retic Perspective, in: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United ...

    2022

  54. [62]

    B. Wang, S. Wang, Y. Cheng, Z. Gan, R. Jia, B. Li, J. Liu, InfoBERT: Improving Robustness of Language Models from An Information Theo- retic Perspective, in: 9th International Conference on Learning Repre- sentations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021, 2021. 27

    2021

  55. [63]

    Z. Chi, L. Dong, F. Wei, N. Yang, S. Singhal, W. Wang, X. Song, X. Mao, H. Huang, M. Zhou, InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training, in: Pro- ceedings of the 2021 Conference of the North American Chapter of the Association for C...

    2021

  56. [64]

    L. Wei, Z. Tan, C. Li, J. Wang, W. Huang, Diff-eRank: A Novel Rank- Based Metric for Evaluating Large Language Models, arXiv preprint arXiv:2401.17139 (2024)

    2024 arXiv

  57. [65]

    R. K. Mahabadi, Y. Belinkov, J. Henderson, Variational informa- tion bottleneck for effective low-resource fine-tuning, arXiv preprint arXiv:2106.05469 (2021)

    2021 arXiv

  58. [66]

    Sorensen, J

    T. Sorensen, J. Robinson, C. M. Rytting, A. G. Shaw, K. J. Rogers, A. P. Delorey, M. Khalil, N. Fulda, D. Wingate, An Information-theoretic Approach to Prompt Engineering Without Ground Truth Labels, in: Proceedings of the 60th Annual Meeting of the Association for Com- putati...

    2022

  59. [67]

    X. Chen, K. He, Exploring simple siamese representation learning, in: Proceedings of the IEEE/CVF conference on computer vision and pat- tern recognition, 2021, pp. 15750–15758

    2021

  60. [68]

    Zhang, J

    X. Zhang, J. Zhao, Y. LeCun, Character-level convolutional networks for text classification, Advances in neural information processing systems 28 (2015)

    2015

  61. [69]

    Wang, Glue: A multi-task benchmark and analysis platform for nat- ural language understanding, arXiv preprint arXiv:1804.07461 (2018)

    A. Wang, Glue: A multi-task benchmark and analysis platform for nat- ural language understanding, arXiv preprint arXiv:1804.07461 (2018)

    2018 arXiv

  62. [70]

    A. Wang, Y. Pruksachatkun, N. Nangia, A. Singh, J. Michael, F. Hill, O. Levy, S. Bowman, Superglue: A stickier benchmark for general- purpose language understanding systems, Advances in neural informa- tion processing systems 32 (2019)

    2019

  63. [71]

    A. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, C. Potts, Learning word vectors for sentiment analysis, in: Proceedings of the 28 49th annual meeting of the association for computational linguistics: Human language technologies, 2011, pp. 142–150

    2011

  64. [72]

    Z. Wang, S. V. Mehta, B. Pocz´ os, J. G. Carbonell, Efficient Meta Lifelong-Learning with Limited Memory, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020, pp. 535–548

    2020

  65. [73]

    Y. Song, P. Wang, W. Xiong, D. Zhu, T. Liu, Z. Sui, S. Li, InfoCL: Al- leviating Catastrophic Forgetting in Continual Text Classification from An Information Theoretic Perspective, in: Findings of the Association for Computational Linguistics: EMNLP 2023, 2023, pp. 14557–14570

    2023

  66. [74]

    Y. Guo, S. Xu, J. Fu, J. Liu, C. Dong, B. Wang, Q-Tuning: Queue-based Prompt Tuning for Lifelong Few-shot Language Learning, in: Findings of the Association for Computational Linguistics: NAACL 2024, 2024, pp. 2595–2622

    2024

  67. [75]

    Bohao, Z

    P. Bohao, Z. Tian, S. Liu, M.-C. Yang, J. Jia, Scalable Language Model with Generalized Continual Learning, in: ICLR 2024

    2024

  68. [76]

    Koehn, Statistical significance tests for machine translation evalua- tion, in: Proceedings of the 2004 conference on empirical methods in natural language processing, 2004, pp

    P. Koehn, Statistical significance tests for machine translation evalua- tion, in: Proceedings of the 2004 conference on empirical methods in natural language processing, 2004, pp. 388–395

    2004

  69. [77]

    Paszke, S

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al., Pytorch: An imper- ative style, high-performance deep learning library, Advances in neural information processing systems 32 (2019)

    2019

  70. [78]

    Wolf, Huggingface’s transformers: State-of-the-art natural language processing, arXiv preprint arXiv:1910.03771 (2019)

    T. Wolf, Huggingface’s transformers: State-of-the-art natural language processing, arXiv preprint arXiv:1910.03771 (2019)

    2019 arXiv

  71. [79]

    D. Rao, F. Visin, A. Rusu, R. Pascanu, Y. W. Teh, R. Hadsell, Continual unsupervised representation learning, Advances in neural information processing systems 32 (2019)

    2019

  72. [80]

    D. P. Kingma, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980 (2014). 29

    2014 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.