Pith. sign in

REVIEW 3 major objections 5 minor 33 references

An institutional platform and open plugins can turn generic AI coding agents into domain-aware collaborators that respect scientific norms and leave auditable trails of technical reasoning.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 07:36 UTC pith:6NB76ZA4

load-bearing objection Useful, concrete institutional stack and open RSE plugin design for a real gap; the 'shift' claim outruns the evidence, but the paper is honest about that. the 3 major comments →

arxiv 2607.02703 v1 pith:6NB76ZA4 submitted 2026-07-02 cs.SE cs.AIcs.DCcs.MA

LLMoxie: Exploring Agentic AI for Scientific Software Development

Landung Setiawan , Anant Mittal , Cordero Core , Anshul Tambay , Carlos Garcia Jurado Suarez , David A. C. Beck , Andrew J. Connolly , Vani Mandava This is my paper
classification cs.SE cs.AIcs.DCcs.MA
keywords agentic AIresearch software engineeringscientific softwarecoding agentsreproducibilityPlugin-Agent-Skillscientific PythonAI governance
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Scientific software is judged less by raw code quality than by whether it can be cited, audited, reproduced, and extended. Off-the-shelf AI coding agents, trained and scored on commercial software, routinely ignore scientific Python conventions, mishandle sensitive or embargoed data, and leave decision trails that are hard to reconstruct later. Drawing on twenty months of practice in a multi-domain research software engineering center, this paper presents LLMoxie: a three-tier platform for multi-cloud and on-premise inference, governance (authentication, budgets, PII masking, observability), and application augmentation. Layered on top is an open RSE-Plugins ecosystem that encodes scientific Python practice, domain knowledge, a six-phase research-and-implement workflow, and project lifecycle management as a Plugin-Agent-Skill hierarchy. The claim is that this stack recalibrates coding agents into collaborators that respect community norms and emit provenance of technical reasoning without fine-tuning the model.

Core claim

Together, the LLMoxie platform and RSE-Plugins ecosystem shift AI coding agents from generic code generators into domain-aware collaborators that respect community norms and produce auditable provenance of technical reasoning. The platform separates inference, governance, and application augmentation; the plugins package accumulated research-software practice as composable context that a coding agent consumes at inference time.

What carries the argument

The Plugin-Agent-Skill hierarchy (hosted by LLMoxie’s three-tier architecture): installable Plugins bundle domain-expert Agents and focused Skills, plus slash commands for a six-phase research-and-implement workflow that writes durable markdown artifacts of research, plans, experiments, implementation, and validation. This encodes RSE craft as inference-time context rather than model retraining.

Load-bearing premise

That packaging research-software knowledge as plugins and skills, consumed at inference time without fine-tuning the model, is enough to make generic coding agents follow scientific conventions and leave useful, reconstructible decision trails on real multi-domain projects.

What would settle it

A controlled comparison on real scientific software tasks in which the same coding agent with and without RSE-Plugins shows no improvement in adherence to community conventions, reconstructibility of design decisions from the six-phase markdown artifacts, or downstream reproducibility and maintainability of the resulting code.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Institutions can offer governed multi-cloud and on-premise AI coding with shared authentication, budgets, PII masking, and usage telemetry without rewriting applications.
  • Coding agents can be steered toward scientific Python packaging, testing, documentation, and domain formats (e.g. FITS, Xarray/NetCDF) at inference time rather than by fine-tuning.
  • The six-phase workflow externalizes technical reasoning so engineering decisions become part of the citable, auditable scientific artifact.
  • Modular plugins let teams adopt domain guidance independently and, in principle, contribute community plugins back into a shared ecosystem.
  • Centralized observability supports cost control and sustainability planning when cloud credits expire.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the markdown decision trails prove reconstructible at scale, funders and journals may eventually expect AI-assisted provenance packages alongside code and data deposits.
  • The same Plugin-Agent-Skill pattern could transfer to other norm-heavy crafts (lab protocols, clinical trial software) where community convention matters more than generic code quality.
  • Without controlled productivity and quality measures, institutions may under-invest because gains remain anecdotal even if the architecture is sound.
  • Long-term value hinges on portability across coding agents; if the stack stays tied to one agent, it risks becoming a vendor-specific tool rather than shared infrastructure.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents LLMoxie, a three-tier institutional AI platform (multi-cloud/on-premise inference; LiteLLM/MLflow control plane for auth, budgeting, PII masking, and observability; application augmentation for coding agents) together with an open-source RSE-Plugins ecosystem that encodes research software engineering practice as a Plugin–Agent–Skill hierarchy. The hierarchy covers scientific Python conventions, domain knowledge (initially astronomy and Earth/climate), a six-phase research-and-implement workflow with durable markdown artifacts, and project lifecycle management. Drawing on twenty months of multi-domain practice at a university RSE center, the authors characterize infrastructure, governance, and process challenges of adopting agentic AI for scientific software, describe the platform and plugin design, and distill operational lessons. The central claim is that this stack shifts off-the-shelf coding agents from generic code generators into domain-aware collaborators that respect community norms and produce auditable provenance of technical reasoning.

Significance. The paper targets a genuine and under-served gap: commercial AI coding agents are poorly calibrated to scientific software norms (reproducibility, specialized formats, sensitive/embargoed data, reconstructible decision trails). The architectural description (Sections 3–5) is concrete—multi-backend routing, governance controls, Pulumi/Azure IaC, and a composable Plugin–Agent–Skill design that avoids fine-tuning—and the open-source framing plus honest Limitations (Section 7) are strengths. If the practice lessons and plugin design hold up under broader use, the work would be a useful systems and process contribution for RSE centers and scientific computing communities. Credit is due for separating inference, governance, and application layers; for encoding Scientific Python Development Guide conventions and Diátaxis documentation practice as reusable skills; and for explicitly scoping future work on intent debt and provenance rather than overstating current auditability.

major comments (3)
  1. [Abstract, §8, §7] Abstract and §8 (and the parallel claim in the introduction): the load-bearing claim that the platform and plugins “shift AI coding agents from generic code generators into domain-aware collaborators that respect community norms and produce auditable provenance of technical reasoning” is not supported by evidence of the kind the claim requires. §7 states there is no controlled user study, no quantitative productivity/quality/reproducibility measures, and no multi-agent portability exercise. Without a baseline comparison (agent alone vs. agent+plugins) on shared scientific tasks, or even systematic artifact review of the .agents/ markdown trails across projects, causal attribution of any observed improvement to the hierarchy remains an assumption. Either temper the claim to “designed to support / intended to shift” and frame the paper as a systems-and-practice report, or add concrete eval
  2. [§4.3, §6] §4.3 and §6: the six-phase workflow is presented as yielding “auditable trails of technical reasoning” and extending reproducibility “to the software engineering decisions themselves.” The mechanism (structured markdown under .agents/, slash commands) is clear, but the paper does not show that later collaborators actually reconstructed decisions from those artifacts, nor how completeness/consistency of the trails was assessed across the multi-domain portfolio. This is central to the provenance half of the main claim; a short case vignette (one project’s research→plan→implement→validate trail and how it was used in handoff or review) would make the claim checkable.
  3. [§1, §4.2] §1 and §4.2: the portfolio is said to span astronomy, earth and climate science, agriculture, and health, yet domain plugins are only instantiated for astronomy/astrophysics and xarray-centric Earth/climate workflows; agriculture and health are “an active workstream.” The multi-domain generalization argument for an RSE-center response therefore rests partly on domains not yet encoded. Either narrow the multi-domain claim to domains with shipped plugins, or document what transferable patterns from agriculture/health already informed the cross-cutting Scientific Python and workflow plugins.
minor comments (5)
  1. [§5, Figure 4] Figure 4 caption and surrounding text refer to “2026-05-13” spend and usage; ensure consistency with the stated practice window (September 2024–May 2026) and that any future-dated analytics do not confuse readers about the evaluation period.
  2. [§5.3, §4.1.3] §5.3 lists flake8/prettier/codespell for quality gates while §4.1.3 promotes ruff as the unified linter/formatter for scientific Python; a brief note reconciling platform vs. plugin tooling choices would reduce apparent inconsistency.
  3. [References] Several references are arXiv preprints or forthcoming items with 2025–2026 dates; verify final bibliographic details and DOIs where available before camera-ready.
  4. [§2] §2: the transition from RAG pilots to agent/skill-centric design is useful background; a short explicit statement of what was discarded vs. retained from the RAG stack would help readers who face the same migration.
  5. [§3.3–§4] Clarify availability: repository URLs, license, and which plugins are installable today versus planned would strengthen the open-source contribution claim in §3.3–§4.

Circularity Check

0 steps flagged

No circular derivation: systems/practice report with no fitted-input predictions, self-definitional reductions, or load-bearing uniqueness claims.

full rationale

LLMoxie is an architecture-and-practice paper, not a first-principles or predictive derivation. The three-tier platform (inference / LiteLLM-MLflow control plane / RSE-Plugins) and the Plugin-Agent-Skill hierarchy are presented as engineered artifacts encoding existing RSE conventions (Scientific Python Development Guide, Diátaxis, Astropy/Xarray patterns, six-phase workflow). There are no equations that reduce a claimed prediction to a fitted parameter, no uniqueness theorem imported from the authors to forbid alternatives, and no ansatz smuggled via self-citation. Self-references (e.g., the earlier RAG pilot [28], NAIRR award [20]) document prior work by the same group; they are not used as the sole external warrant for a mathematical result. The strongest claim—that the stack shifts agents into domain-aware collaborators—is an interpretive summary of twenty months of multi-domain practice, explicitly caveated in Limitations (no controlled study, no quantitative productivity/quality metrics). That is an evidence-strength issue, not circularity by construction. Score 0 is therefore the correct, proportionate finding.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 3 invented entities

This is an engineering experience paper, not a formal derivation. Load-bearing content consists of domain premises about scientific software quality criteria, the premise that inference-time context (plugins) can substitute for fine-tuning, and the invented system entities (platform tiers and plugin hierarchy) whose value is asserted from practice rather than independent external measurement.

axioms (4)
  • domain assumption Scientific software is judged less by raw code quality than by whether it can be cited, audited, reproduced, and extended.
    Stated in abstract and Introduction as the distinctive evaluation criterion that commercial coding benchmarks miss; underpins the entire motivation for domain-aware plugins.
  • domain assumption Off-the-shelf AI coding agents optimized on commercial benchmarks ignore scientific Python conventions, mishandle sensitive/embargoed data, and leave hard-to-reconstruct decision trails.
    Core problem statement in abstract and Section 1; supported by citations but treated as given for the design response.
  • ad hoc to paper Encoding accumulated RSE craft as composable Plugin-Agent-Skill context at inference time is preferable to (and sufficient without) model retraining or fine-tuning for scientific calibration.
    Explicit design philosophy in Section 4; the platform's value proposition rests on this choice rather than on a comparative experiment against fine-tuned models.
  • ad hoc to paper A structured six-phase research-and-implement workflow with durable markdown artifacts yields auditable provenance of technical reasoning as a routine byproduct.
    Section 4.3; central to the claim of decision transparency; not independently validated with user studies in the paper.
invented entities (3)
  • LLMoxie three-tiered architecture (inference + LiteLLM/MLflow control plane + application augmentation) no independent evidence
    purpose: Provide governed multi-cloud/on-premise inference with auth, budgeting, PII masking, observability, and a surface for domain plugins.
    The paper's primary system contribution; value is demonstrated by internal deployment description and usage snapshot, not by external independent measurement.
  • RSE-Plugins Plugin-Agent-Skill hierarchy (Scientific Python, Domain Applications, AI Research Workflows, Project Management plugins) no independent evidence
    purpose: Encode scientific Python practice, domain knowledge, six-phase workflow, and lifecycle management as reusable context for coding agents.
    The application-layer invention that is claimed to convert generic agents into domain-aware collaborators; independent external validation is not provided.
  • Six-phase research-and-implement workflow with /research /plan /iterate-plan /experiment /implement /validate commands no independent evidence
    purpose: Impose structured methodology and emit durable decision artifacts for auditability.
    Key mechanism for the provenance claim; effectiveness rests on practitioner observation.

pith-pipeline@v1.1.0-grok45 · 17944 in / 3199 out tokens · 38083 ms · 2026-07-12T07:36:27.327879+00:00 · methodology

0 comments
read the original abstract

In this paper, we describe LLMoxie, an institutional AI platform whose three-tiered architecture supports multi-cloud and on-premise inference, a LiteLLM/MLflow control plane for authentication, budgeting, PII masking, and observability, and an application augmentation layer for AI coding agents. Layered on top, an open-source RSE-Plugins ecosystem encodes accumulated RSE knowledge as a Plugin-Agent-Skill hierarchy spanning scientific Python practice, domain-specific knowledge, a six-phase research-and-implement workflow, and project lifecycle management. Scientific software is judged less by raw code quality than by whether it can be cited, audited, reproduced, and extended. Off-the-shelf AI coding agents, optimized against commercial software benchmarks, are poorly calibrated for this setting: they ignore the conventions of the scientific Python libraries they invoke, mishandle sensitive or embargoed data, and leave decision trails that are difficult to reconstruct after the fact. We report on twenty months of practice at a university-based research software engineering (RSE) center, where RSEs embedded across astronomy, earth and climate science, agriculture, and health projects worked to close this gap. We characterize the recurring infrastructure, governance, and process challenges of adopting Agentic AI inside a multi-domain RSE center, describe the platform and plugin design, and distill operational lessons from real scientific software deployments. Together, the platform and plugins shift AI coding agents from generic code generators into domain-aware collaborators that respect community norms and produce auditable provenance of technical reasoning.

Figures

Figures reproduced from arXiv: 2607.02703 by Anant Mittal, Andrew J. Connolly, Anshul Tambay, Carlos Garcia Jurado Suarez, Cordero Core, David A. C. Beck, Landung Setiawan, Vani Mandava.

Figure 1
Figure 1. Figure 1: An RSE-oriented scientific software development lifecycle. Conventional engineering stages (Spec, Dev, Test, Ship) are [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: LLMoxie’s three-tiered architecture for AI-enabled [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: LLMoxie Azure deployment architecture. Pulumi [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: LLMoxie Analytics snapshot. LLMoxie platform [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

33 extracted references · 3 canonical work pages

  1. [1]

    Peter Belcak, Greg Heinrich, Shizhe Diao, Yonggan Fu, Xin Dong, Saurav Muralidharan, Yingyan Celine Lin, and Pavlo Molchanov. 2025. Small Language Models are the Future of Agentic AI. doi:10.48550/arXiv.2506.02153

    Pith/arXiv arXiv doi:10.48550/arxiv.2506.02153 2025

  2. [2]

    Besser, Eric A

    Stephanie A. Besser, Eric A. Jensen, and Daniel S. Katz. 2026. How Generative AI Is Shaping Research Software Development and Maintenance at a Research- Intensive University.Open Research Europe6 (2026), 56. doi:10.12688/ openreseurope.22009.1

    2026

  3. [3]

    Bridgeford, Iain Campbell, Zijao Chen, Zhicheng Lin, Harrison Ritz, Joachim Vandekerckhove, and Russell A

    Eric W. Bridgeford, Iain Campbell, Zijao Chen, Zhicheng Lin, Harrison Ritz, Joachim Vandekerckhove, and Russell A. Poldrack. 2025. Ten Simple Rules for AI-Assisted Coding in Science. doi:10.48550/arXiv.2510.22254

    doi:10.48550/arxiv.2510.22254 2025

  4. [4]

    Databricks. 2024. Long Context RAG Performance of LLMs. https://www. databricks.com/blog/long-context-rag-performance-llms

    2024

  5. [5]

    Titouan Duston, Shuo Xin, Yang Sun, Daoguang Zan, Aoyan Li, Shulin Xin, Kai Shen, Yixiao Chen, Qiming Sun, Ge Zhang, Jiashuo Liu, Huan Zhou, Jingkai Liu, Zhichen Pu, Yuanheng Wang, Bo-Xuan Ge, Xin Tong, Fei Ye, Zhi-Chao Zhao, Wen-Biao Han, Zhoujian Cao, Yueran Zhao, Weiluo Ren, Qingshen Long, Yuxiao Liu, Anni Huang, Yidi Du, Yuanyuan Rong, and Jiahao Peng...

    arXiv 2025

  6. [6]

    Siamak Farshidi, Kwabena Bennin, Önder Babur, June Sallou, Ayalew Kassahun, and Bedir Tekinerdogan. 2025. Advancing Research Software Engineering With AI: A Research Framework. doi:10.21203/rs.3.rs-7178452/v1

    work page doi:10.21203/rs.3.rs-7178452/v1 2025

  7. [8]

    Sandra Gesing. 2025. RSEs 2035: Surviving or Thriving in the Age of AI. In 2025 IEEE International Conference on eScience (eScience). 381–382. doi:10.1109/ eScience65000.2025.00081 ISSN: 2325-3703

    arXiv 2025

  8. [9]

    Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, A. Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Raghavi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Daniel Morrison, Niklas Muennighoff, Aakanksh...

    2024

  9. [10]

    Alexandre Hocquet, Frédéric Wieber, Gabriele Gramelsberger, Konrad Hinsen, Markus Diesmann, Fernando Pasquini Santos, Catharina Landström, Benjamin Peters, Dawid Kasprowicz, Arianna Borrelli, Phillip Roth, Clarissa Ai Ling Lee, Alin Olteanu, and Stefan Böschen. 2024. Software in Science Is Ubiquitous yet Overlooked.Nature Computational Science(July 2024),...

    doi:10.1038/s43588- 2024

  10. [11]

    Mohammad Hosseini, Serge P J M Horbach, Kristi L Holmes, and Tony Ross- Hellauer. 2024. Open Science at the Generative AI Turn: An Exploratory Analysis of Challenges and Opportunities. doi:10.1162/qss_a_00337

    doi:10.1162/qss_a_00337 2024

  11. [12]

    Alyssa Hughes. 2022. AI4Science to Empower the Fifth Paradigm of Scientific Discovery. https://www.microsoft.com/en-us/research/blog/ai4science-to- empower-the-fifth-paradigm-of-scientific-discovery/

    2022

  12. [13]

    Vásquez, Richard Barnes, and Ciera C

    Haley Hunter-Zinck, Alexandre Fioravante De Siqueira, Váleri N. Vásquez, Richard Barnes, and Ciera C. Martinez. 2021. Ten Simple Rules on Writing Clean and Reliable Open-Source Scientific Software.PLOS Computational Biology17, 11 (Nov. 2021), e1009481. doi:10.1371/journal.pcbi.1009481

    doi:10.1371/journal.pcbi.1009481 2021

  13. [14]

    Sebastian Antony Joseph, Syed Murtaza Husain, Stella S. R. Offner, Stéphanie Juneau, Paul Torrey, Adam S. Bolton, Juan P. Farias, Niall Gaffney, Greg Durrett, and Junyi Jessy Li. 2025. AstroVisBench: A Code Benchmark for Scientific Computing and Visualization in Astronomy. (2025). doi:10.48550/ARXIV.2505. 20538

    doi:10.48550/arxiv.2505 2025

  14. [15]

    Udo-Imeh, Bonan Kou, and Tianyi Zhang

    Samia Kabir, David N. Udo-Imeh, Bonan Kou, and Tianyi Zhang. 2024. Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI ’24). Association for Computing Machinery, New York, NY, USA, 1–17. doi:10.1145/3613904.3642596

    doi:10.1145/3613904.3642596 2024

  15. [16]

    Sayash Kapoor, Benedikt Stroebl, Peter Kirgis, Nitya Nadgir, Zachary S. Siegel, Boyi Wei, Tianci Xue, Ziru Chen, Felix Chen, Saiteja Utpala, Franck Ndzomga, Dheeraj Oruganty, Sophie Luskin, Kangheng Liu, Botao Yu, Amit Arora, Dongyoon Hahm, Harsh Trivedi, Huan Sun, Juyong Lee, Tengjun Jin, Yifan Mai, Yifei Zhou, Yuxuan Zhu, Rishi Bommasani, Daniel Kang, D...

    arXiv 2025

  16. [17]

    Miklós Koren, Gábor Békés, Julian Hinz, and Aaron Lohmann. 2026. Vibe Coding Kills Open Source. doi:10.48550/arXiv.2601.15494

    doi:10.48550/arxiv.2601.15494 2026

  17. [18]

    Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, and Jennifer Neville. 2025. LLMs Get Lost In Multi-Turn Conversation. doi:10.48550/arXiv.2505.06120

    Pith/arXiv arXiv doi:10.48550/arxiv.2505.06120 2025

  18. [19]

    Hao Li, Yiqun Zhang, Zhaoyan Guo, Chenxu Wang, Shengji Tang, Qiaosheng Zhang, Yang Chen, Biqing Qi, Peng Ye, Lei Bai, Zhen Wang, and Shuyue Hu

  19. [20]

    LLMRouterBench: A Massive Benchmark and Unified Framework for LLM Routing. (2026). doi:10.48550/ARXIV.2601.07206

    doi:10.48550/arxiv.2601.07206 2026

  20. [21]

    Vani Mandava. [n. d.]. NAIRR Current Projects: NAIRR240292. https://submit- nairr.xras.org/current-projects?_requestNumber=NAIRR240292. Accessed: 2026- 06-28

    2026

  21. [22]

    Newton, Chris Oehmen, Stefan M

    Lois Curfman McInnes, Dorian Arnold, Prasanna Balaprakash, Mike Bernhardt, Beth Cerny, Anshu Dubey, Roscoe Giles, Denice Ward Hood, Mary Ann Leung, Vanessa Lopez-Marrero, Paul Messina, Olivia B. Newton, Chris Oehmen, Stefan M. Wild, Jim Willenbring, Lou Woodley, Tony Baylis, David E. Bernholdt, Chris Camano, Johannah Cohoon, Charles Ferenbaugh, Stephen M....

    doi:10.48550/arxiv.2510 2025

  22. [23]

    Linda Nordling. 2023. How ChatGPT Is Transforming the Postdoc Experience. Nature622, 7983 (Oct. 2023), 655–657. doi:10.1038/d41586-023-03235-8

    work page doi:10.1038/d41586-023-03235-8 2023

  23. [24]

    Gabrielle O’Brien. 2025. How Scientists Use Large Language Models to Program. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Association for Computing Machinery, New York, NY, USA, 1–16. doi:10.1145/3706598.3713668

    doi:10.1145/3706598.3713668 2025

  24. [25]

    Gabrielle O’Brien, Alexis Parker, Nasir Eisty, and Jeffrey Carver. 2026. A Survey of Generative AI Adoption and Perceived Productivity Among Scientists Who Program. doi:10.48550/arXiv.2512.19644

    Pith/arXiv arXiv doi:10.48550/arxiv.2512.19644 2026

  25. [26]

    Gabrielle O’Brien. 2025. Threats to Scientific Software From Over-Reliance on AI Code Assistants.Nature Computational Science5, 9 (Sept. 2025), 701–703. doi:10.1038/s43588-025-00845-2

    doi:10.1038/s43588-025-00845-2 2025

  26. [27]

    Deepak Babu Piskala. 2026. Spec-Driven Development:From Code to Contract in the Age of AI Coding Assistants. doi:10.48550/arXiv.2602.00180 arXiv:2602.00180 [cs.SE]

    doi:10.48550/arxiv.2602.00180 2026

  27. [28]

    rOpenSci. 2026. Software Review in the Era of AI: What We Are Testing at rOpenSci. (Feb. 2026). doi:10.59350/5tavw-mke71

    work page doi:10.59350/5tavw-mke71 2026

  28. [29]

    Don Setiawan, Anant Mittal, Anuj Sinha, Anuj Khandelwal, Cordero Core, Madhav Kashyap, Niki Burggraf, Anshul Tambay, and Vani Mandava. 2024. Generative AI Copilot for Scientific Software – a RAG-Based Approach. https:// uw-ssec-tutorials.readthedocs.io/en/latest/Archive/SciPy2024/README.html

    2024

  29. [30]

    Judy Hanwen Shen and Alex Tamkin. 2026. How AI Impacts Skill Formation. doi:10.48550/arXiv.2601.20245

    doi:10.48550/arxiv.2601.20245 2026

  30. [31]

    Margaret-Anne Storey. 2026. From Technical Debt to Cognitive and Intent Debt: Rethinking Software Health in the Age of AI. doi:10.48550/arXiv.2603.22106

    Pith/arXiv arXiv doi:10.48550/arxiv.2603.22106 2026

  31. [32]

    Minyang Tian, Luyu Gao, Shizhuo Dylan Zhang, Xinan Chen, Cunwei Fan, Xuefei Guo, Roland Haas, Pan Ji, Kittithat Krongchon, Yao Li, Shengyan Liu, Di Luo, Yutao Ma, Hao Tong, Kha Trinh, Chenyu Tian, Zihan Wang, Bohao Wu, Yanyu Xiong, Shengzhu Yin, Minhui Zhu, Kilian Lieret, Yanxin Lu, Genglin Liu, Yufeng Du, Tianhua Tao, Ofir Press, Jamie Callan, Eliu Huert...

  32. [33]

    SciCode: A Research Coding Benchmark Curated by Scientists. (2024). doi:10.48550/ARXIV.2407.13168

    Pith/arXiv arXiv doi:10.48550/arxiv.2407.13168 2024

  33. [34]

    Suqing Wu, Yukun Liu, Mengqi Ruan, Siyu Chen, and Xiao-Yun Xie. 2025. Human- Generative AI Collaboration Enhances Task Performance but Undermines Human’s Intrinsic Motivation.Scientific Reports15, 1 (April 2025), 15105. doi:10.1038/s41598-025-98385-2

    doi:10.1038/s41598-025-98385-2 2025