Pith. sign in

REVIEW 4 major objections 7 minor 54 references

Agent Turns Scattered Research Software Chats into Shared Memory

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

2026-07-10 01:04 UTC pith:CBQ6WQ24

load-bearing objection Well-motivated system design for a real RSE problem, but the core value proposition is entirely unmeasured. the 4 major comments →

arxiv 2607.08043 v1 pith:CBQ6WQ24 submitted 2026-07-09 cs.SE cs.AI

Aleena: Alignment Agent for Research Software Engineering Collaborations

Kshitij Dani , Cordero Core , Landung Setiawan , Carlos Garcia Jurado Suarez , Anshul Tambay , Vani Mandava , Anant Mittal This is my paper
classification cs.SE cs.AI
keywords researchsoftwarealignmentaleenaengineeringlifecycleagentcollaborations
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Research software collaborations suffer from a chronic alignment problem: decisions originate in Slack threads, get refined in meetings, and land in pull requests, but the rationale connecting these artifacts dissolves across modalities. Domain researchers and research software engineers are left with divergent mental models of project intent, ownership, and scientific assumptions. This paper argues that alignment in research software engineering is not a one-time scoping activity but a continuous lifecycle problem that must be actively maintained. The authors present Aleena, an open-source agent that treats GitHub as a shared collaboration surface and operates across multi-modal project artifacts—meeting transcripts, chat logs, issues, and pull requests. Aleena runs a four-step loop: it perceives submitted artifacts, updates an evolving project-state model with structured signals (action items, risks, open questions, terminology-drift candidates, ownership transitions), selects a GitHub-native action (open issue, start discussion, draft pull request), and then defers—all merges, closures, and external actions remain under human control. The central claim is that by transforming scattered interactions into structured, reviewable GitHub records that persist across the project lifecycle, an AI agent can support stakeholder alignment without replacing human decision-making.

Core claim

The paper reframes research software alignment as a continuous project-state management problem and demonstrates a mechanism for it: an agentic loop that ingests multi-modal collaboration artifacts, extracts structured alignment signals, and writes them back as GitHub-native records (issues, discussions, draft pull requests) while preserving human authority over every decision. The four-step Perceive-Update-Select-Defer loop is the load-bearing structure, and the project-state model organized around four alignment challenges—stakeholder expectations, scientific vocabulary, ownership responsibilities, and decision continuity—is the object that carries the argument.

What carries the argument

Aleena's architecture is a five-layer loop. Layer 1 captures meeting and chat transcripts submitted by stakeholders. Layer 2 transforms them into structured GitHub artifacts using task-specific analyzer modules, each with an editable prompt, routed through a multi-provider LLM gateway. Layer 3 produces context-aware suggestions and draft pull requests grounded in prior interaction context. Layer 4 routes outputs through stakeholder review. Layer 5 closes the loop by updating the project-state model ahead of the next interaction. The four-step agentic loop (Perceive, Update, Select, Defer) governs each cycle: perceive ingests artifacts with retrieved project history; update extracts and merge

Load-bearing premise

The paper assumes that LLM-generated structured artifacts—meeting summaries, risk flags, terminology-drift signals, ownership assignments—will be accurate and useful enough that busy stakeholders will actually review, retain, and act on them rather than ignoring or dismissing them. The entire value proposition depends on sustained human engagement with these generated records, and this engagement has not yet been measured.

What would settle it

If stakeholders routinely close Aleena's generated GitHub artifacts without action, edit them so heavily that the original signal is lost, or stop submitting artifacts because the outputs are not useful, then the system produces noise rather than alignment and the core thesis fails. The paper explicitly acknowledges this by identifying retention, edit, and close-without-action rates as the key future evaluation metrics.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the approach works, research software engineering centers could systematically preserve decision rationale across short engagements, reducing the institutional memory loss that plagues 3–6 month collaborative projects.
  • The Perceive-Update-Select-Defer pattern, where an agent writes structured records to a shared surface but never executes irreversible actions, could generalize to other multi-stakeholder domains beyond research software—any setting where alignment erodes across communication modalities.
  • If stakeholders routinely retain Aleena's outputs as-is or with edits, it would validate the thesis that LLM-extracted structured artifacts are useful coordination primitives; if they close them without action, the system produces noise rather than alignment.
  • The terminology-drift analyzer, which detects when the same word carries different meanings in scientific versus engineering contexts, could become a general tool for boundary-spanning work in interdisciplinary collaborations.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. This paper presents Aleena, an open-source lifecycle alignment agent designed to support stakeholder alignment in research software engineering (RSE) collaborations. The system operates across multi-modal project artifacts (meeting transcripts, chat logs, GitHub issues, pull requests) and transforms them into structured GitHub records—summaries, risks, open questions, terminology-drift signals, and draft pull requests—while preserving human decision-making authority. The paper identifies four recurring alignment challenges (AC1–AC4) grounded in the authors' experience at a university-based RSE center, describes a five-layer architecture with a four-step agentic loop (Perceive, Update, Select, Defer), and illustrates the system's behavior through three lifecycle scenarios. The prototype is deployed and seeing early operational use, though no empirical evaluation is yet available.

Significance. The paper addresses a genuine and well-motivated problem: alignment in research software collaborations is distributed across heterogeneous artifacts and modalities, and the rationale behind decisions is frequently lost. The framing of alignment as a continuous lifecycle problem rather than a one-time scoping activity is a useful conceptual contribution. The system design is architecturally coherent: the four-step loop with a hard 'Defer' step that prevents autonomous merges or closures is a principled design choice that addresses legitimate concerns about AI overreach in collaborative settings. The open-source, GitHub-native approach is practical and portable. The inclusion of editable prompts (Appendix A) and the transparent acknowledgment that evaluation is pending are commendable. However, the paper's central claim—that Aleena supports stakeholder alignment—remains asserted rather than demonstrated, which significantly limits the current contribution's evidential weight.

major comments (4)
  1. [Section 3 ('Deployment status and scope') and Section 3 ('Future evaluation')] The paper's central claim is that Aleena 'supports stakeholder alignment and project-state tracking without replacing human decision-making.' This claim is load-bearing and entirely unvalidated. Section 3 explicitly states the system is 'building history to evaluate and measure... what fraction of those artifacts stakeholders retain as-is, edit before acting on, or close without action,' and the 'Future evaluation' paragraph confirms these metrics are uncollected. The three scenarios in Section 4 are illustrative, not empirical. Without any data on whether stakeholders actually engage with, retain, or act on Aleena's outputs, the paper cannot distinguish between a system that supports alignment and one that generates noise that exacerbates the very artifact fragmentation it aims to solve. The paper is transparent about this gap, but the gap is the entire difference between a system描述 and
  2. [Section 3, 'project-state model'] The 'project-state model' is described as central to Aleena's operation—it is 'organized around AC1–AC4' and is the substrate into which structured signals are merged (Update step) and from which actions are selected (Select step). However, its structure, representation, and update semantics are not specified. Is it a database schema, a knowledge graph, a set of GitHub labels and linked issues, or something else? How are conflicting signals reconciled when new artifacts arrive? Without this specification, it is difficult to assess whether the system can actually maintain coherent state across a multi-month project, or whether the 'project-state model' is an aspirational label for what is currently unstructured LLM output written to GitHub artifacts.
  3. [Section 4, Scenario 2 (AC2, AC3) and Appendix A.3] The vocabulary-drift detection scenario claims that Aleena surfaces 'conflicting interpretations' of terms like 'model' across artifacts. Appendix A.3 shows the drift-detection prompt compares two definitions and classifies them as 'duplicate' or 'conflict.' However, the paper does not describe how term occurrences are extracted from artifacts in the first place (the vocabulary extraction prompt in A.2 operates on meeting transcripts only), how definitions are associated with occurrences, or how the system handles the common case where a term is used informally without an explicit definition. The scenario presents a clean case (a term with two explicit, conflicting definitions) but does not address whether this reflects realistic artifact content. A brief discussion of precision/recall expectations or a concrete example with actual system output would strengthen this claim.
  4. [Section 2 and Ref [10]] The four alignment challenges (AC1–AC4) are grounded in 'a retrospective activity following the delivery of more than 20 projects' (Section 2), citing Ref [10], which is authored by center members (Dailey and Tambay). This is self-reported and not independently validated. While this does not invalidate the challenges, the paper would benefit from either (a) providing more detail about the retrospective methodology (how data was collected, what the 20 projects spanned, how challenges were identified and categorized) or (b) connecting AC1–AC4 to prior literature on alignment challenges in software engineering more explicitly, so that the problem framing does not rest solely on an internal report. The current connection to prior work (design rationale capture [7, 9, 22], shared mental models [11, 41], boundary objects [30, 45]) is useful but does not directly map to AC1–AC4.
minor comments (7)
  1. [Section 1] The phrase 'two directions of teaching occur in parallel' is a useful framing but is introduced without explicit connection to the alignment challenges. A brief forward reference to AC2 (vocabulary and domain assumptions) would help the reader see the connection.
  2. [Figure 1] The bottom row summarizing Aleena's role per phase is too small to read clearly. Consider enlarging or splitting into a table.
  3. [Section 3, 'Positioning relative to other platforms'] The comparison to OpenClaw, Hermes, and Paperclip is useful but these systems are described very briefly. A sentence or two on each system's actual capabilities would help readers unfamiliar with them.
  4. [Section 5] The discussion of 'offboarding memory' is introduced in one sentence and then dropped. A brief elaboration on what form this memory would take (a summary document? a knowledge graph export?) would help.
  5. [Appendix A.1] The meeting-minutes prompt instructs the model to 'never return an empty object' and 'extract at least the most important concrete points.' These instructions may conflict in edge cases (e.g., very short or off-topic transcripts). Consider adding guidance for handling degenerate inputs.
  6. [Section 3] The claim that Aleena 'has begun seeing early operational use on a number of active collaborative software projects' is vague. Even a rough count (e.g., 'three projects' or 'five projects') would be informative.
  7. [References] Several references have 2026 dates (e.g., [3], [32], [35], [40]) which appear to be forward-dated. Verify these are correct and not placeholder dates.

Circularity Check

0 steps flagged

No circularity: system design paper with no mathematical derivation chain or first-principles predictions

full rationale

This is a system design and experience paper, not a derivation paper. The paper's chain is: (1) identify alignment challenges AC1–AC4 from a retrospective activity (ref [10], a self-authored center retrospective), (2) design Aleena's four-step loop (Perceive, Update, Select, Defer) to address them, (3) implement a prototype, (4) illustrate with lifecycle scenarios. No step reduces to its inputs by construction. There are no equations, no fitted parameters presented as predictions, no uniqueness theorems invoked, and no ansatz smuggled through self-citation. The self-citation [10] grounds the problem motivation in the center's own experience, but it is not load-bearing for any mathematical claim—it is a retrospective learning document, not a theorem or verified result being used to forbid alternatives. The paper's claims are design claims (the agent 'supports alignment without replacing human decision-making'), explicitly acknowledged as unmeasured in Section 3 ('Deployment status and scope') and Section 3 ('Future evaluation'). The absence of empirical validation is a correctness/evaluation concern, not a circularity concern. The derivation chain is self-contained and non-circular.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 2 invented entities

The paper introduces two main invented entities (project-state model, terminology-drift signal) and relies on three ad-hoc assumptions, the most critical being that LLM outputs are accurate enough and that stakeholders will engage with them. These are not independently validated in the paper.

free parameters (2)
  • LLM model selection
    The paper does not specify which LLM models are used for each analyzer task. The LiteLLM gateway routes requests based on availability, privacy, and cost, but the specific model choices and their impact on output quality are not reported.
  • Prompt engineering parameters
    The prompts in Appendix A include specific instructions (e.g., '2-5 sentences for summary', 'at least the most important concrete points') that were presumably tuned by the authors. No systematic prompt optimization is described.
axioms (4)
  • domain assumption Alignment in research software engineering is a continuous lifecycle problem rather than a one-time scoping activity.
    Stated in the Abstract and Introduction. This is the foundational framing assumption. It is supported by the retrospective activity (ref 10) but is an interpretive claim, not a measured result.
  • ad hoc to paper LLM-generated structured artifacts (summaries, risks, terminology signals) are sufficiently accurate to be useful for stakeholder alignment.
    This is assumed throughout Section 3 and Section 4 but never empirically validated. The prompts in Appendix A produce JSON output, but no accuracy metrics are reported.
  • ad hoc to paper Stakeholders will review, edit, or act on Aleena's GitHub outputs rather than ignoring them.
    The entire value proposition depends on this. The paper acknowledges it is 'building history to evaluate' this (Section 3, Deployment status).
  • domain assumption GitHub is an appropriate shared collaboration surface for research software teams.
    Stated in Section 3. This is reasonable for software engineering but may not fully capture teams that primarily use other platforms.
invented entities (2)
  • Project-state model no independent evidence
    purpose: An evolving internal representation of alignment concerns (AC1-AC4) that Aleena maintains across artifacts.
    The project-state model is referenced in Section 3 as the core data structure Aleena maintains, but its implementation, schema, and update mechanism are not specified in detail. No falsifiable handle is provided.
  • Terminology-drift signal no independent evidence
    purpose: A ranked list of candidate terms with conflicting definitions across artifacts, detected by comparing definitions using the vocabulary drift prompt (Appendix A.3).
    The drift detection mechanism is described conceptually and the comparison prompt is provided, but no evaluation of its precision or recall is reported.

pith-pipeline@v1.1.0-glm · 14686 in / 2844 out tokens · 307533 ms · 2026-07-10T01:04:06.628856+00:00 · methodology

0 comments
read the original abstract

Research software collaborations span meetings, informal chats, pull requests, and GitHub issues. A decision surfaced in a Slack thread, refined in a meeting, and implemented in a pull request can lose its original rationale across these artifacts, leaving domain researchers and research software engineers with divergent mental models of project intent, ownership, and scientific assumptions. We argue that alignment in research software engineering is a continuous lifecycle problem, and that agentic AI can support stakeholder alignment and project-state tracking without replacing human decision-making. We present Aleena, an open-source lifecycle alignment agent that uses GitHub as a shared collaboration surface, transforming multi-modal stakeholder interactions into structured project records that surface risks, track open questions, and preserve decision continuity. Grounded in university-based research software engineering center experiences, this paper presents the motivating problem, system design, prototype, and illustrative lifecycle scenarios for Aleena.

Figures

Figures reproduced from arXiv: 2607.08043 by Anant Mittal, Anshul Tambay, Carlos Garcia Jurado Suarez, Cordero Core, Kshitij Dani, Landung Setiawan, Vani Mandava.

Figure 1
Figure 1. Figure 1: The RSE project lifecycle at and Aleena’s role across it. The top row shows the five phases (Pre-Selection, Selection, Scoping and Design, Implementation, Offboarding); the bottom row summarizes how Aleena supports each phase. Together they ground the four alignment challenges AC1– AC4 that recur throughout the lifecycle. become the primary artifacts through which direction and scope is refined, but the re… view at source ↗
Figure 2
Figure 2. Figure 2: End-to-end Aleena workflow, organized as five layers that move from interaction artifacts to project-state updates and back. Layer 1 captures meeting and chat transcripts; Layer 2 transforms them into structured GitHub artifacts (meeting summaries, issues, risks, and discussions); Layer 3 produces context-aware suggestions and draft pull requests grounded in prior interaction context; Layer 4 routes these … view at source ↗
Figure 3
Figure 3. Figure 3: The current Aleena prototype interface. Stakeholders sign in with GitHub, upload a meeting document or paste a chat transcript, confirm the target repository, and select which analyses to run. Each analysis exposes an editable prompt so that researchers and RSEs can inspect and adjust how Aleena interprets the artifact before it produces GitHub-native outputs. A live run log makes the agent’s progress obse… view at source ↗
Figure 5
Figure 5. Figure 5: shows an example of a draft PR generated by Aleena based on an interaction. In this scenario, the owner of the PR reviews, approves, and manually merges the PR [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 4
Figure 4. Figure 4: shows an example of how Aleena creates a GitHub Discussion that records decisions made during a specific engagement using the prompt in Appendix A.1 [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

54 extracted references · 54 canonical work pages · 5 internal anchors

  1. [1]

    Elvira-Maria Arvanitou, Apostolos Ampatzoglou, Alexander Chatzigeorgiou, and Jeffrey C. Carver. 2021. Software Engineering Practices for Scientific Software Development: A Systematic Mapping Study.Journal of Systems and Software172 (Feb. 2021), 110848. doi:10.1016/j.jss.2020.110848

    work page doi:10.1016/j.jss.2020.110848 2021

  2. [2]

    Sumit Asthana, Sagi Hilleli, Pengcheng He, and Aaron Halfaker. 2025. Summaries, Highlights, and Action Items: Design, Implementation and Evaluation of an LLM- Powered Meeting Recap System.Proceedings of the ACM on Human-Computer Interaction9, 2 (May 2025), CSCW176:1–CSCW176:29. doi:10.1145/3711074

    work page doi:10.1145/3711074 2025

  3. [3]

    Atlassian. 2026. Jira. https://www.atlassian.com/software/jira

    work page 2026

  4. [4]

    Adrian Bajraktari, Michelle Binder, and Andreas Vogelsang. 2024. Requirements Engineering for Research Software: A Vision. In2024 IEEE 32nd International Requirements Engineering Conference (RE). IEEE, Reykjavik, Iceland, 423–431. doi:10.1109/RE59067.2024.00050

    work page doi:10.1109/re59067.2024.00050 2024

  5. [5]

    Peter Belcak, Greg Heinrich, Shizhe Diao, Yonggan Fu, Xin Dong, Saurav Muralidharan, Yingyan Celine Lin, and Pavlo Molchanov. 2025. Small Language Models are the Future of Agentic AI. doi:10.48550/arXiv.2506.02153

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2506.02153 2025

  6. [6]

    Bridgeford, Iain Campbell, Zijao Chen, Zhicheng Lin, Harrison Ritz, Joachim Vandekerckhove, and Russell A

    Eric W. Bridgeford, Iain Campbell, Zijao Chen, Zhicheng Lin, Harrison Ritz, Joachim Vandekerckhove, and Russell A. Poldrack. 2025. Ten Simple Rules for AI-Assisted Coding in Science. doi:10.48550/arXiv.2510.22254

    work page doi:10.48550/arxiv.2510.22254 2025

  7. [7]

    Burge and David C

    Janet E. Burge and David C. Brown. 2008. Software Engineering Using RATionale. Journal of Systems and Software81, 3 (March 2008), 395–413. doi:10.1016/j.jss. 2007.05.004

    work page doi:10.1016/j.jss 2008

  8. [8]

    Jeffrey C. Carver. 2012. Software Engineering for Computational Science and Engineering.Computing in Science & Engineering14, 02 (March 2012), 8–11. doi:10.1109/MCSE.2012.31

    work page doi:10.1109/mcse.2012.31 2012

  9. [9]

    Jeff Conklin and Michael L. Begeman. 1988. gIBIS: A Hypertext Tool for Exploratory Policy Discussion. InProceedings of the 1988 ACM conference on Computer-supported cooperative work (CSCW ’88). Association for Computing Machinery, New York, NY, USA, 140–152. doi:10.1145/62266.62278

    work page doi:10.1145/62266.62278 1988

  10. [10]

    Dharma Dailey and Anshul Tambay. 2024. Selected Year One Learnings of the Scientific Software Engineering Center

    work page 2024

  11. [11]

    Alberto Espinosa, Sandra A

    J. Alberto Espinosa, Sandra A. Slaughter, Robert E. Kraut, and James D. Herbsleb

    work page

  12. [12]

    2007), 613–630

    Familiarity, Complexity, and Team Performance in Geographically Distributed Software Development.Organization Science18, 4 (Aug. 2007), 613–630. doi:10.1287/orsc.1070.0297

    work page doi:10.1287/orsc.1070.0297 2007

  13. [13]

    Tao Feng, Haozhen Zhang, Zijie Lei, Haodong Yue, Chongshan Lin, Ge Liu, and Jiaxuan You. 2025. LLMRouter: An Open-Source Library for LLM Routing. https://github.com/ulab-uiuc/LLMRouter

    work page 2025

  14. [14]

    Tiantian Gan and Qiyao Sun. 2025. RAG-MCP: Mitigating Prompt Bloat in LLM Tool Selection via Retrieval-Augmented Generation. doi:10.48550/arXiv.2505. 03275

    work page doi:10.48550/arxiv.2505 2025

  15. [15]

    Sandra Gesing. 2025. RSEs 2035: Surviving or Thriving in the Age of AI. In 2025 IEEE International Conference on eScience (eScience). 381–382. doi:10.1109/ eScience65000.2025.00081 ISSN: 2325-3703

    work page arXiv 2025

  16. [16]

    Logan Golia and Jugal Kalita. 2024. Action-Item-Driven Summarization of Long Meeting Transcripts. InProceedings of the 2023 7th International Conference on Natural Language Processing and Information Retrieval (NLPIR ’23). Association Dani et al. for Computing Machinery, New York, NY, USA, 91–98. doi:10.1145/3639233. 3639253

    work page doi:10.1145/3639233 2024

  17. [18]

    Jo Erskine Hannay, Carolyn MacLeod, Janice Singer, Hans Petter Langtangen, Dietmar Pfahl, and Greg Wilson. 2009. How Do Scientists Develop and Use Scientific Software?. In2009 ICSE Workshop on Software Engineering for Computational Science and Engineering. 1–8. doi:10.1109/SECSE.2009.5069155

    work page doi:10.1109/secse.2009.5069155 2009

  18. [19]

    Hideaki Hata, Nicole Novielli, Sebastian Baltes, Raula Gaikovina Kula, and Christoph Treude. 2022. GitHub Discussions: An Exploratory Study of Early Adoption.Empirical Software Engineering27, 1 (Jan. 2022), 3. doi:10.1007/s10664- 021-10058-6

    work page doi:10.1007/s10664- 2022

  19. [20]

    Dustin Heaton and Jeffrey C. Carver. 2015. Claims About the Use of Software Engineering Practices in Science: A Systematic Literature Review.Information and Software Technology67 (Nov. 2015), 207–219. doi:10.1016/j.infsof.2015.07.011

    work page doi:10.1016/j.infsof.2015.07.011 2015

  20. [21]

    Alexandre Hocquet, Frédéric Wieber, Gabriele Gramelsberger, Konrad Hinsen, Markus Diesmann, Fernando Pasquini Santos, Catharina Landström, Benjamin Peters, Dawid Kasprowicz, Arianna Borrelli, Phillip Roth, Clarissa Ai Ling Lee, Alin Olteanu, and Stefan Böschen. 2024. Software in Science Is Ubiquitous yet Overlooked.Nature Computational Science(July 2024),...

    work page doi:10.1038/s43588- 2024

  21. [22]

    Lennart Hofeditz, Milad Mirbabaie, and Mara Ortmann. 2024. Ethical Challenges for Human–Agent Interaction in Virtual Collaboration at Work.International Journal of Human–Computer Interaction40, 23 (Dec. 2024), 8229–8245. doi:10. 1080/10447318.2023.2279400

    work page arXiv 2024

  22. [23]

    John Horner and M. E. Atwood. 2006. Effective Design Rationale: Understanding the Barriers. InRationale Management in Software Engineering, Allen H. Dutoit, Raymond McCall, Ivan Mistrík, and Barbara Paech (Eds.). Springer, Berlin, Heidelberg, 73–90. doi:10.1007/978-3-540-30998-7_3

    work page doi:10.1007/978-3-540-30998-7_3 2006

  23. [24]

    Mohammad Hosseini, Serge P J M Horbach, Kristi L Holmes, and Tony Ross- Hellauer. 2024. Open Science at the Generative AI Turn: An Exploratory Analysis of Challenges and Opportunities. doi:10.1162/qss_a_00337

    work page doi:10.1162/qss_a_00337 2024

  24. [25]

    Vásquez, Richard Barnes, and Ciera C

    Haley Hunter-Zinck, Alexandre Fioravante De Siqueira, Váleri N. Vásquez, Richard Barnes, and Ciera C. Martinez. 2021. Ten Simple Rules on Writing Clean and Reliable Open-Source Scientific Software.PLOS Computational Biology17, 11 (Nov. 2021), e1009481. doi:10.1371/journal.pcbi.1009481

    work page doi:10.1371/journal.pcbi.1009481 2021

  25. [26]

    Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R

    Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R. Narasimhan. 2023. SWE-Bench: Can Language Models Resolve Real-World GitHub Issues? https://openreview.net/forum? id=VTF8yNQM66

    work page 2023

  26. [27]

    Arne Johanson and Wilhelm Hasselbring. 2018. Software Engineering for Computational Science: Past, Present, Future.Computing in Science & Engineering 20, 2 (March 2018), 90–109. doi:10.1109/MCSE.2018.021651343

    work page doi:10.1109/mcse.2018.021651343 2018

  27. [28]

    Udo-Imeh, Bonan Kou, and Tianyi Zhang

    Samia Kabir, David N. Udo-Imeh, Bonan Kou, and Tianyi Zhang. 2024. Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI ’24). Association for Computing Machinery, New York, NY, USA, 1–17. doi:10.1145/3613904.3642596

    work page doi:10.1145/3613904.3642596 2024

  28. [29]

    Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, and Jennifer Neville. 2025. LLMs Get Lost In Multi-Turn Conversation. doi:10.48550/arXiv.2505.06120

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2505.06120 2025

  29. [30]

    Paperclip Labs. [n. d.]. Paperclip – The App People Use to Manage AI Agents for Work. https://paperclip.ing/

    work page

  30. [31]

    Charlotte P. Lee. 2007. Boundary Negotiating Artifacts: Unbinding the Routine of Boundary Objects and Embracing Chaos in Collaborative Work.Computer Supported Cooperative Work (CSCW)16, 3 (June 2007), 307–339. doi:10.1007/ s10606-007-9044-5

    work page 2007

  31. [32]

    Hao Li, Haoxiang Zhang, and Ahmed E. Hassan. 2025. The Rise of AI Teammates in Software Engineering (SE) 3.0: How Autonomous Coding Agents Are Reshaping Software Engineering. doi:10.48550/arXiv.2507.15003

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2507.15003 2025

  32. [33]

    Hao Li, Yiqun Zhang, Zhaoyan Guo, Chenxu Wang, Shengji Tang, Qiaosheng Zhang, Yang Chen, Biqing Qi, Peng Ye, Lei Bai, Zhen Wang, and Shuyue Hu

    work page

  33. [34]

    LLMRouterBench: A Massive Benchmark and Unified Framework for LLM Routing. (2026). doi:10.48550/ARXIV.2601.07206

    work page doi:10.48550/arxiv.2601.07206 2026

  34. [35]

    Newton, Chris Oehmen, Stefan M

    Lois Curfman McInnes, Dorian Arnold, Prasanna Balaprakash, Mike Bernhardt, Beth Cerny, Anshu Dubey, Roscoe Giles, Denice Ward Hood, Mary Ann Leung, Vanessa Lopez-Marrero, Paul Messina, Olivia B. Newton, Chris Oehmen, Stefan M. Wild, Jim Willenbring, Lou Woodley, Tony Baylis, David E. Bernholdt, Chris Camano, Johannah Cohoon, Charles Ferenbaugh, Stephen M....

    work page doi:10.48550/arxiv.2510 2025

  35. [36]

    Rob van Nieuwpoort and Daniel S. Katz. 2023. Defining the Roles of Research Software.Upstream(March 2023). doi:10.54900/9akm9y5-5ject5y

    work page doi:10.54900/9akm9y5-5ject5y 2023

  36. [37]

    Notion Labs, Inc. 2026. Notion. https://www.notion.com/

    work page 2026

  37. [38]

    Gabrielle O’Brien. 2025. How Scientists Use Large Language Models to Program. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Association for Computing Machinery, New York, NY, USA, 1–16. doi:10.1145/3706598.3713668

    work page doi:10.1145/3706598.3713668 2025

  38. [39]

    Gabrielle O’Brien, Alexis Parker, Nasir Eisty, and Jeffrey Carver. 2026. A Survey of Generative AI Adoption and Perceived Productivity Among Scientists Who Program. doi:10.48550/arXiv.2512.19644

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2512.19644 2026

  39. [40]

    Gabrielle O’Brien. 2025. Threats to Scientific Software From Over-Reliance on AI Code Assistants.Nature Computational Science5, 9 (Sept. 2025), 701–703. doi:10.1038/s43588-025-00845-2

    work page doi:10.1038/s43588-025-00845-2 2025

  40. [41]

    Raphael Pham, Stephan Kiesling, Leif Singer, and Kurt Schneider. 2017. Onboarding Inexperienced Developers: Struggles and Perceptions Regarding Automated Testing.Software Quality Journal25, 4 (Dec. 2017), 1239–1268. doi:10.1007/s11219-016-9333-7

    work page doi:10.1007/s11219-016-9333-7 2017

  41. [42]

    Nous Research. 2026. Hermes Agent. https://github.com/NousResearch/hermes- agent

    work page 2026

  42. [43]

    Cannon-Bowers Eduardo Salas and Sharolyn Converse

    Janis A. Cannon-Bowers Eduardo Salas and Sharolyn Converse. 1993. Shared Mental Models in Expert Team Decision Making*. InIndividual and Group Decision Making. Psychology Press. Num Pages: 26

    work page 1993

  43. [44]

    Rebecca Sanders and Diane Kelly. 2008. Dealing With Risk in Scientific Software Development.IEEE Software25, 4 (July 2008), 21–28. doi:10.1109/MS.2008.84

    work page doi:10.1109/ms.2008.84 2008

  44. [45]

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language Models Can Teach Themselves To Use Tools. https: //openreview.net/forum?id=Yacmpz84TH

    work page 2023

  45. [46]

    Judy Hanwen Shen and Alex Tamkin. 2026. How AI Impacts Skill Formation. doi:10.48550/arXiv.2601.20245

    work page doi:10.48550/arxiv.2601.20245 2026

  46. [47]

    Griesemer

    Susan Leigh Star and James R. Griesemer. 1989. Institutional Ecology, ‘Translations’ and Boundary Objects: Amateurs and Professionals in Berkeley’s Museum of Vertebrate Zoology, 1907–39.Social Studies of Science19, 3 (Aug. 1989), 387–420. doi:10.1177/030631289019003001

    work page doi:10.1177/030631289019003001 1989

  47. [48]

    Peter Steinberger. [n. d.]. OpenClaw — Personal AI Assistant. https://openclaw. ai/

    work page

  48. [49]

    Margaret-Anne Storey. 2026. From Technical Debt to Cognitive and Intent Debt: Rethinking Software Health in the Age of AI. doi:10.48550/arXiv.2603.22106

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2603.22106 2026

  49. [50]

    Margaret-Anne Storey and Alexey Zagalsky. 2016. Disrupting Developer Productivity One Bot at a Time. InProceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE 2016). Association for Computing Machinery, New York, NY, USA, 928–931. doi:10. 1145/2950290.2983989

    work page arXiv 2016

  50. [51]

    Viktoria Stray and Nils Brede Moe. 2020. Understanding Coordination in Global Software Engineering: A Mixed-Methods Study on the Use of Meetings and Slack. Journal of Systems and Software170 (Dec. 2020), 110717. doi:10.1016/j.jss.2020. 110717

    work page doi:10.1016/j.jss.2020 2020

  51. [52]

    Jason Tsay, Laura Dabbish, and James Herbsleb. 2014. Influence of Social and Technical Factors for Evaluating Contribution in GitHub. InProceedings of the 36th International Conference on Software Engineering (ICSE 2014). Association for Computing Machinery, New York, NY, USA, 356–366. doi:10.1145/2568225. 2568315

    work page doi:10.1145/2568225 2014

  52. [53]

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Ji-Rong Wen. 2024. A Survey on Large Language Model Based Autonomous Agents.Frontiers of Computer Science18, 6 (Dec. 2024), 186345. doi:10.1007/ s11704-024-40231-1

    work page 2024

  53. [54]

    Suqing Wu, Yukun Liu, Mengqi Ruan, Siyu Chen, and Xiao-Yun Xie. 2025. Human- Generative AI Collaboration Enhances Task Performance but Undermines Human’s Intrinsic Motivation.Scientific Reports15, 1 (April 2025), 15105. doi:10.1038/s41598-025-98385-2

    work page doi:10.1038/s41598-025-98385-2 2025

  54. [55]

    summary

    John Yang, Carlos Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering.Advances in Neural Information Processing Systems37 (Dec. 2024), 50528–50652. doi:10.52202/079017-1601 Aleena: Alignment Agent for Research Software Engineering Collabo...

    work page doi:10.52202/079017-1601 2024