Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

SWE-Bench-CL: Continual Learning for Coding Agents

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SWE-Bench-CL claims that chronologically reordering verified GitHub issues into per-repository sequences lets researchers directly measure whether coding agents accumulate experience, transfer knowledge, and resist forgetting, and…

desk verdict A useful new benchmark dataset for continual learning in coding agents, but the proposed evaluation framework's memory design confounds the forgetting metric it claims to measure. read the letter →

arxiv 2507.00014 v1 pith:DHKKTGVD submitted 2025-06-13 cs.LG cs.AIcs.SE

Thomas Joshi , Shayan Chowdhury , Fatih Uysal This is my paper
classification cs.LGcs.AIcs.SE
keywords continuallearningcodingagentsSWE-Benchbenchmarkcatastrophicforgettingsemanticmemoryretrieval-augmentedgenerationstability-plasticity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that coding agents should be evaluated on a stream of real repository issues rather than isolated tasks, and introduces SWE-Bench-CL, a benchmark that reorders human-verified GitHub issues into chronological, difficulty-ordered sequences per repository. The claim is that this temporal structure makes it possible to measure whether an agent accumulates experience, transfers knowledge to new tasks, and retains old skills instead of forgetting them. To support that measurement, the paper defines a suite of continual-learning metrics around a performance matrix $a_{i,j}$, including accuracy, forgetting, forward and backward transfer, tool-use efficiency, and a stability-plasticity composite score. It also reports that the standard static SWE-Bench harness is misaligned with the derived temporal dataset and proposes an interactive agentic evaluation framework with semantic memory. If the benchmark is valid, it gives the field a direct way to compare how quickly and how stably agents learn from a codebase's history.

What carries the argument

The load-bearing object is the performance matrix $a_{i,j}$ over an ordered sequence of $N$ repository tasks, where $a_{i,j}$ is the success rate on task $j$ after the agent has processed task $i$. The dataset construction carries the benchmark: 273 verified issues from 8 Python repositories, each repository's issues sorted by creation time and then by estimated fix difficulty, with modified-file metadata to flag potential dependencies. The metrics defined on this matrix carry the argument: average accuracy, forgetting, forward transfer, backward transfer, area under the learning curve, tool-use efficiency, and a weighted composite CL-Score. The stability-plasticity trade-off is captured by CL-F1 and its generalized CL-F$\beta$ form, the harmonic mean of CL-Plasticity and CL-Stability. The proposed evaluation agent combines an interactive tool-use loop with a semantic memory index that stores past task summaries and retrieves them for new tasks, enabling direct comparison of memory-enabled and memory-disabled agents.

What would settle it

Re-run the SWE-Bench-CL protocol on one sequence and check, for each earlier task, whether its FAIL_TO_PASS and PASS_TO_PASS tests can still be executed at that task's base commit after later tasks have been processed; if any re-test fails for environmental reasons, the $a_{i,j}$ matrix and all metrics derived from it contain artifacts rather than measurements of forgetting.

Watch

Extended reading notes

Core claim

The central claim is that restructuring the human-verified SWE-Bench task set into chronologically ordered per-repository sequences turns a static code-fixing benchmark into a continual-learning benchmark whose metrics separate immediate proficiency from retention and transfer. In this construction, each task stays anchored at its own base commit, the ground-truth patch and test lists are reused from the verified dataset, and tasks are ordered first by creation time and then by human-estimated fix difficulty. The paper defines the performance matrix $a_{i,j}$ as the success rate on task $j$ after the agent has processed task $i$, and derives from it CL-Plasticity, CL-Stability, and the composite CL-F1 score that balances them. The same matrix feeds average accuracy, forgetting, forward and backward transfer, and area-under-learning-curve. The paper presents these metrics as the quantitative language for assessing continual learning in software engineering, and reports preliminary evidence that irrelevant retrieved context can shift generated solutions substantially.

Load-bearing premise

The forgetting and transfer metrics assume that an earlier task can be re-tested later with the agent's memory intact, but each task is run at its own base commit and the paper does not explain how old repository states are reconstructed or replayed as the codebase evolves.

Editorial extensions

If this is right

  • If the matrix is a faithful measurement, an agent's SWE-Bench-CL score separates 'can solve this issue now' from 'remembered how to solve earlier issues', which static pass@k cannot do.
  • The prompt-poisoning result implies that memory-augmented agents must gate retrieved context; prepending unrelated past issue-patch pairs produced average semantic drift around 0.45.
  • The reported harness incompatibility implies that reusing the original SWE-Bench evaluation tooling on the chronologically reordered derived tasks can produce unreliable near-zero pass rates, motivating a dedicated interactive harness.
  • If the proposed protocol is run as hypothesized, memory-enabled agents should show higher CL-Stability and better CL-F1 than memory-disabled agents, while memory's effect on immediate accuracy may be smaller.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same chronological structure could be used for fine-tuning curricula, not just evaluation: sequentially fine-tuning a model on repository history and measuring the same matrix would test whether the model itself retains fixes rather than relying on an external memory.
  • Because tasks are anchored at different base commits, the matrix assumes replay of old environments; a natural extension is to specify and validate environment-reconstruction rules before reporting forgetting scores.
  • The drift result suggests a testable design principle for memory modules: retrieval quality should be judged by whether including a memory changes the final patch, not only by similarity to the current issue.
  • The benchmark's curriculum ordering could be compared against shuffled or anti-curriculum orders to test whether the difficulty ramp causally improves forward transfer, a question the paper motivates but does not resolve.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces SWE-Bench-CL, a continual learning benchmark derived from SWE-Bench Verified by organizing GitHub issues from eight Python repositories (273 tasks total) into sequences. The authors claim this temporal structuring enables direct evaluation of an agent's ability to accumulate experience, transfer knowledge, and resist catastrophic forgetting. The paper describes dataset construction, reports a preliminary analysis of inter-task similarity and prompt-poisoning sensitivity, proposes a LangGraph-based agentic evaluation framework with a FAISS semantic memory module, and defines a suite of continual learning metrics (ACC, F, FT, BWT, AULC, TUE, CL-Score, CL-Fβ). Full empirical evaluation is stated to be ongoing in Section 8.

Significance. If the benchmark and metrics are valid, the paper would fill a real gap: static code benchmarks do not measure retention, transfer, or forgetting in coding agents. The dataset construction is transparent and reproducible, the use of standard continual learning definitions (e.g., forward/backward transfer, average forgetting) is appropriate, and the public code release is a strength. The preliminary analyses are useful but limited. However, the central claim that the framework enables direct evaluation of catastrophic forgetting is currently undermined by how the memory system and the re-testing protocol interact, and no empirical results exist to validate the proposed metrics. The contribution is therefore promising but not yet established.

major comments (4)
  1. [§6.3, §7 (F and CL-S)] The definitions of average forgetting F and CL-S in Section 7 assume that a_{i,j} is the success rate on task j after learning task i, obtained by 'testing on all previously seen tasks after each new issue.' However, Section 6.3 stores in the FAISS index, for every completed attempt, summaries of the problem, solution, and rationale. When task j is re-tested after a later task i, the memory-enabled agent can retrieve and replay the stored summary of task j's own solution; a_{N,j} then reflects retrieval fidelity rather than retention of learned skill. Consequently, CL-S (1 − F) is confounded by direct answer lookup, and the stability-plasticity trade-off in Section 7.1 is not measured for the proposed agent as claimed in the abstract.
  2. [§6, §8] For the memory-disabled condition, the paper describes no parameter updates: the agent performs fixed-weight inference with a growing (or absent) memory. Catastrophic forgetting is a phenomenon of learners whose parameters or internal representations are updated by new tasks; a frozen model has no mechanism for catastrophic forgetting, so F is trivially near zero. The planned comparison between memory-enabled and memory-disabled agents therefore cannot demonstrate resistance to catastrophic forgetting in the sense the central claim asserts. A learning mechanism (e.g., fine-tuning, adapter updates, or replay-based training) must be specified for both arms, or the claim about forgetting should be removed.
  3. [§6.4, §7] The performance matrix a_{i,j} requires that each previously seen task j be re-tested after each subsequent task i, yet Section 6.4 states that every task is run at its own base_commit. The paper does not specify how an old task's environment is reconstructed or replayed after the repository has evolved to a later base_commit, nor how the agent's memory state from later tasks is applied to the older checkout. Without a concrete replay protocol, the entries a_{i,j} for i > j are not well-defined as stated, and the forgetting and transfer metrics built on them are not measurable from the described workflow.
  4. [§3.2] The ordering procedure is ambiguous or contradictory: tasks are said to be 'primarily ordered by their creation timestamp' and then 'further ordered by difficulty,' presenting easier tasks first. If difficulty reorders the sequence, the resulting order does not reflect natural repository evolution as advertised; if the chronological order is primary, the difficulty-based curriculum cannot be implemented globally. The paper needs to state the exact sorting rules, including whether the difficulty ordering is applied within chronological windows or overrides timestamps, because the central claim of temporal realism depends on this.
minor comments (5)
  1. [§7, Composite CL-Score formula] The formula for CL-Score appears to have a typographical inconsistency: it reads 'CL-Score = ACC−λ F F+λ F TFT +λ BW TBWT +λ AU LCAULC + CL−Fβ', which lacks explicit plus signs between all terms and omits the λ_TUE weight that is described in the table below. Please reformat the equation and align the symbol list with the weight table.
  2. [§7.1, CL-Fβ] The CL-Fβ score is defined solely from CL-P and CL-S, which are already components of the composite CL-Score. This is a redundancy rather than a circularity, but the paper should clarify whether CL-Fβ is intended as an alternative summary or an additional term in CL-Score; including it twice could inflate its influence on the composite.
  3. [§4.2, Figure 2] The paper says differences between target difficulty groups were 'not always statistically significant due to sample sizes,' but no sample sizes, p-values, or effect-size statistics are reported for the prompt-poisoning experiment. Please provide these details so readers can judge the strength of the drift findings.
  4. [§5, Figures 3 and 4] The empirical results with the standard SWE-Bench harness are reported as very low pass rates and high Levenshtein distances, but the figures lack confidence intervals or per-model breakdowns. Since these results motivate the proposed agentic framework, more detail on the number of tasks, models, and variance would strengthen the motivation.
  5. [Abstract and §8] The abstract states that the benchmark 'enables direct evaluation' of accumulation, transfer, and forgetting, while Section 8 discloses that full empirical evaluation is ongoing. Consider tempering the abstract to say the benchmark is 'designed to enable' such evaluation, until the metrics are validated with the proposed framework.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the benchmark construction and metric definitions are self-contained; the composite score contains redundancy but no prediction reduces to its inputs.

full rationale

SWE-Bench-CL is built by reordering externally sourced SWE-Bench Verified instances into per-repository sequences (Section 3.2). No quantity used to construct the sequences is derived from the metrics the paper proposes, and no parameter is fitted to a subset of data and then reported as a prediction. The metrics in Section 7 are explicit definitions of standard continual-learning quantities (average accuracy, forgetting, forward/backward transfer, AULC, TUE), with user-set weights; CL-P and CL-S are defined directly from the performance matrix, and CL-Fβ is their harmonic mean. The Composite CL-Score includes CL-Fβ alongside F and CL-P, so forgetting is double-counted, but this is redundancy, not circularity: it does not make any claimed result equivalent to an input by construction. Two measurement-validity concerns are worth stating but are not circularity. First, Section 6.3 stores per-task summaries of problem, solution, and rationale in a FAISS index; when Section 7 re-tests an earlier task after later tasks, a memory-enabled agent can retrieve that task's own stored solution, so a_{N,j} may reflect retrieval fidelity rather than retained skill. Second, Section 8's memory-disabled condition uses fixed-weight inference with no parameter updates, so catastrophic forgetting is arguably undefined for that arm. These are threats to metric validity, not reductions of the claimed evaluation to its inputs. There is no load-bearing self-citation chain and no imported uniqueness theorem; citations to SWE-Bench, SWE-agent, and FAISS are external and do not carry the central argument. Under the definition of circularity used here, the paper warrants a score of 0.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The benchmark itself is the product, not an invented entity. The free parameters are tunable weights and an arbitrary threshold, none fitted to data. The key assumptions are domain-level: the validity of the source benchmark, the curriculum signal, and the feasibility of the performance matrix.

free parameters (3)
  • CL-Score weights lambda_F, lambda_FT, lambda_BWT, lambda_AULC, lambda_TUE = user-specified (no defaults given)
    Tunable weights balance forgetting penalty and transfer/learning speed rewards in the composite score; chosen by the experimenter, not fitted to data.
  • beta in CL-Fbeta = user-specified (beta=1 for balanced)
    Controls emphasis on plasticity vs. stability in the F-score; user-chosen.
  • High drift threshold 0.3 = 0.3
    Arbitrary cutoff in Figure 2 to label high semantic drift; influences interpretation of prompt-poisoning results.
assumptions (5)
  • domain assumption SWE-Bench Verified is a valid human-verified benchmark for real GitHub issue resolution
    Used as the source and ground truth for all sequences (Section 3.2).
  • domain assumption Chronological creation-date ordering approximates a natural developer learning trajectory
    Basis for sequence construction; acknowledged as simplified in Section 9.
  • domain assumption Human fix-time categories are a valid difficulty curriculum signal
    Used for curriculum ordering (Table 1); authors note subjectivity in Section 9.
  • domain assumption The a_{i,j} matrix can be computed by re-testing all previous tasks after each new task
    Foundation of all CL metrics (Section 7); protocol for replaying old environments with evolving memory is not specified.
  • domain assumption Cosine similarity of TF-IDF/embeddings of patches reflects task relatedness
    Used in Section 4.1 to argue low inter-task similarity and in Section 4.2 to measure drift.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SWE-Bench-CL: Continual Learning for Coding Agents." pith.science (2026). https://pith.science/paper/DHKKTGVD

@misc{pith2026250700014,
  author       = {Pith},
  title        = {Pith review of: SWE-Bench-CL: Continual Learning for Coding Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DHKKTGVD}},
  note         = {Machine review of arXiv:2507.00014}
}
read the original abstract

Large Language Models (LLMs) have achieved impressive results on static code-generation benchmarks, but real-world software development unfolds as a continuous stream of evolving issues, fixes, and feature requests. We introduce SWE-Bench-CL, a novel continual learning benchmark built on the human-verified SWE-Bench Verified dataset introduced by OpenAI and Princeton-NLP in 2024. By organizing GitHub issues into chronologically ordered sequences that reflect natural repository evolution, SWE-Bench-CL enables direct evaluation of an agent's ability to accumulate experience, transfer knowledge across tasks, and resist catastrophic forgetting. We complement the dataset with (i) a preliminary analysis of inter-task structural similarity and contextual sensitivity, (ii) an interactive LangGraph-based evaluation framework augmented with a FAISS-backed semantic memory module, and (iii) a suite of specialized continual learning metrics -- including average accuracy, forgetting, forward/backward transfer, tool-use efficiency, and a generalized Composite Continual Learning Score and CL-F-beta score -- to capture the stability-plasticity trade-off. We outline a rigorous experimental protocol comparing memory-enabled and memory-disabled agents across diverse Python repositories. All code and data are publicly available at https://github.com/thomasjoshi/agents-never-forget, providing the community with a reproducible platform for developing more adaptive and robust AI agents in software engineering.

Figures

Figures reproduced from arXiv: 2507.00014 by the authors.

Figure 1
Figure 1. Distribution of Patch-Level Similarity Across Tasks in SWE-Bench-CL. Left: Jaccard [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Mean semantic drift induced by prompt poisoning across SWE-Bench-CL tasks, grouped by [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Overall Pass Rate on SWE-Bench-CL tasks using the standard SWE-Bench harness. Low pass rates highlight mismatch with static tooling [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ExplainBench: Evaluating Code Explanations from Agents

    cs.SE 2026-07 conditional novelty 6.0 of 10

    Agent explanation quality can be scored automatically with LLM questionnaires, and this score ranks agents differently from SWE-bench Verified.

  2. How Do AI Coding Agents Contribute to Software Development? an Empirical Study of Agentic Pull Requests

    cs.SE 2026-07 conditional novelty 4.0 of 10

    AI coding agents mostly handle routine, well-scoped development tasks; their pull requests are merged at similar rates and have comparable or lower bug-proneness than human-written pull requests across repository lifecycles.

Reference graph

Works this paper leans on

27 extracted references · 6 canonical work pages · cited by 2 Pith papers

  1. [1]

    Program synthesis with large language models, 2021

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. Program synthesis with large language models, 2021. URL https://arxiv.org/abs/2108.07732

    arXiv 2021

  2. [2]

    Curriculum learning

    Yoshua Bengio, Jerome Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th International Conference on Machine Learning (ICML), 2009

    work page 2009

  3. [3]

    Dokania, Thalaiyasingam Ajanthan, and Philip H

    Arslan Chaudhry, Puneet K. Dokania, Thalaiyasingam Ajanthan, and Philip H. S. Torr. Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence, page 556–572. Springer International Publishing, 2018. ISBN 9783030012526. doi:10.1007/978-3-030-01252-6_33. URL http://dx.doi.org/10.1007/978-3-030-01252-6_33

    doi:10.1007/978-3-030-01252-6_33 2018

  4. [4]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian...

    arXiv 2021

  5. [5]

    Deepseek-v3 technical report, 2025

    DeepSeek-AI et al. Deepseek-v3 technical report, 2025. URL https://arxiv.org/abs/2412.19437

    arXiv 2025

  6. [6]

    A continual learning survey: Defying forgetting in classification tasks

    Matthias Delange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Greg Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence, page 1–1, 2021. ISSN 1939-3539. doi:10.1109/tpami.2021.3057446. URL http://dx.doi.org/10.1109/TPAM...

    arXiv 2021

  7. [7]

    The faiss library

    Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. The faiss library. 2024

    2024

  8. [8]

    Gemma 3 technical report, 2025

    Gemma-Team et al. Gemma 3 technical report, 2025. URL https://arxiv.org/abs/2503.19786

    arXiv 2025

Show all 27 references
  1. [9]

    How does a brain build a cognitive code? Studies of mind and brain: Neural principles of learning, perception, development, cognition, and motor control, pages 1--52, 1982

    Stephen Grossberg and Stephen Grossberg. How does a brain build a cognitive code? Studies of mind and brain: Neural principles of learning, perception, development, cognition, and motor control, pages 1--52, 1982

    1982

  2. [10]

    Codesearchnet challenge: Evaluating the state of semantic code search, 2020

    Hamel Husain, Ho-Hsiang Wu, Tiferet Gazit, Miltiadis Allamanis, and Marc Brockschmidt. Codesearchnet challenge: Evaluating the state of semantic code search, 2020. URL https://arxiv.org/abs/1909.09436

    2020 arXiv

  3. [11]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas ...

    2023 arXiv

  4. [12]

    Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan

    Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. Swe-bench: Can language models resolve real-world github issues?, 2024. URL https://arxiv.org/abs/2310.06770

    2024 arXiv

  5. [13]

    Emergent mechanisms for long timescales depend on training curriculum and affect performance in memory tasks, 2024

    Sina Khajehabdollahi, Roxana Zeraati, Emmanouil Giannakakis, Tim Jakob Schäfer, Georg Martius, and Anna Levina. Emergent mechanisms for long timescales depend on training curriculum and affect performance in memory tasks, 2024. URL https://arxiv.org/abs/2309.12927

    2024 arXiv

  6. [14]

    Langchain-ai/langgraph: Build resilient language agents as graphs., 2024

    Langchain. Langchain-ai/langgraph: Build resilient language agents as graphs., 2024. URL https://github.com/langchain-ai/langgraph

    2024

  7. [15]

    Retrieval-augmented generation for knowledge-intensive nlp tasks, 2021

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive nlp tasks, 2021. URL https://arx...

    2021 arXiv

  8. [16]

    Generalization of medical large language models through cross-domain weak supervision, 2025

    Robert Long, Eric Gonzalez, and Harrison Fuller. Generalization of medical large language models through cross-domain weak supervision, 2025. URL https://arxiv.org/abs/2502.00832

    2025 arXiv

  9. [17]

    Gradient episodic memory for continual learning, 2022

    David Lopez-Paz and Marc'Aurelio Ranzato. Gradient episodic memory for continual learning, 2022. URL https://arxiv.org/abs/1706.08840

    2022 arXiv

  10. [18]

    Codexglue: A machine learning benchmark dataset for code understanding and generation, 2021

    Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin Clement, Dawn Drain, Daxin Jiang, Duyu Tang, Ge Li, Lidong Zhou, Linjun Shou, Long Zhou, Michele Tufano, Ming Gong, Ming Zhou, Nan Duan, Neel Sundaresan, Shao Kun Deng, Shengyu Fu, and Shuj...

    2021 arXiv

  11. [19]

    The stability-plasticity dilemma: Investigating the continuum from catastrophic forgetting to age-limited learning effects, 2013

    Martial Mermillod, Aur \'e lia Bugaiska, and Patrick Bonin. The stability-plasticity dilemma: Investigating the continuum from catastrophic forgetting to age-limited learning effects, 2013

    2013

  12. [20]

    Codegen: An open large language model for code with multi-turn program synthesis, 2023

    Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. Codegen: An open large language model for code with multi-turn program synthesis, 2023. URL https://arxiv.org/abs/2203.13474

    2023 arXiv

  13. [21]

    Morris, Brandon Duderstadt, and Andriy Mulyar

    Zach Nussbaum, John X. Morris, Brandon Duderstadt, and Andriy Mulyar. Nomic embed: Training a reproducible long context text embedder, 2025. URL https://arxiv.org/abs/2402.01613

    2025 arXiv

  14. [22]

    Introducing swe-bench verified, 2024

    OpenAI and Princeton-NLP. Introducing swe-bench verified, 2024. URL https://openai.com/index/introducing-swe-bench-verified

    2024

  15. [23]

    Parisi, Ronald Kemker, Jose L

    German I. Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural Networks, 113: 0 54--71, 2019. ISSN 0893-6080. doi:https://doi.org/10.1016/j.neunet.2019.01.012. URL https://www.sciencedire...

    2019 doi

  16. [24]

    Code llama: Open foundation models for code

    Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950, 2023

    2023 arXiv

  17. [25]

    Efficient reinforcement finetuning via adaptive curriculum learning, 2025

    Taiwei Shi, Yiyang Wu, Linxin Song, Tianyi Zhou, and Jieyu Zhao. Efficient reinforcement finetuning via adaptive curriculum learning, 2025. URL https://arxiv.org/abs/2504.05520

    2025 arXiv

  18. [26]

    Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press

    John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. Swe-agent: Agent-computer interfaces enable automated software engineering, 2024. URL https://arxiv.org/abs/2405.15793

    2024 arXiv

  19. [27]

    React: Synergizing reasoning and acting in language models, 2023

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models, 2023. URL https://arxiv.org/abs/2210.03629

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.