Pith. sign in

REVIEW 4 major objections 4 minor 30 references

The Impact of AI-Generated Solutions on Software Architecture and Productivity: Results from a Survey Study

T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read AI tools raise software engineers' productivity, but only when problems are broken into small, well-scoped pieces.

desk verdict The paper has a real survey dataset and sensible qualitative findings, but its headline claim of an 'overwhelming majority' seeing over 35% productivity gains is contradicted by its own reported counts. read the letter →

arxiv 2506.17833 v1 pith:AIFEBIVR submitted 2025-06-21 cs.SE

Giorgio Amasanti , Jasmin Jahic This is my paper · ORCID
classification cs.SE
keywords softwarearchitectureAIproductivityqualityerosionsurveystudyAI-generatedcodeengineering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether AI coding tools deliver real productivity gains without quietly degrading software architecture. Based on a survey of 40 practitioners who already use AI tools, the authors conclude that AI tools significantly increase productivity—most respondents reported gains at or above 35%—and that adopting small AI-generated code snippets does not seriously erode architectural quality. The benefits, however, shrink as problems grow: large AI-generated blocks are harder to maintain, more likely to break existing code, and judged of significantly lower quality than human-written code. The practical conclusion is that AI pays off when engineers decompose problems into small prompts and integrate the results themselves.

What carries the argument

The organizing mechanism is a size-and-complexity boundary rather than a single identity. The study's machinery is the Goal-Question-Metric (GQM) framework used to structure both the Tetris substudy and the survey, combined with grounded-theory open and axial coding of free-text responses. These tools generate the paper's central contrast: small, focused AI-generated snippets preserve cohesion and maintainability, while large AI-generated blocks degrade logical organization. The 35% productivity reference value comes from one engineer's Tetris substudy and anchors the survey's ordinal productivity scale.

What would settle it

Conduct a controlled experiment with two matched teams on the same greenfield and maintenance tasks, one using AI tools and one not, measuring actual cycle time, defect rates, and architectural metrics; if the AI team's measured productivity gain is not significantly above zero, the paper's central claim would not survive.

Watch

Extended reading notes

Core claim

The paper's central claim is that AI assistance is a net productivity positive in software engineering, but only under decomposition: the same tools that generate excellent small snippets produce structurally weak large blocks. The authors arrive at this from a two-part study: a three-week Tetris project built in three parallel versions (no AI, AI pair, multi-agent AI setup), which produced the 35% productivity reference, and a survey of 40 industry practitioners. On self-reported productivity, 79% of respondents matched or exceeded the 35% reference, and 45% said their gains exceeded 35%. On architecture, the survey found no significant erosion of cohesion, coupling, or logical partitioning when adopting small AI-generated snippets, but clear agreement that large AI-generated blocks are of significantly lesser quality, harder to maintain, and more prone to breaking existing code. The authors read this as evidence that architects remain essential: their job is to decompose problems and integrate AI-generated solutions.

Load-bearing premise

The productivity conclusion rests on taking the self-reported 'subjective feeling of productivity' of 40 self-selected practitioners who already use AI tools as a valid measure of real productivity, with no control group and no objective output metric.

Editorial extensions

If this is right

  • Engineers who use AI tools can expect the largest productivity gains in early-stage work such as generating skeletons, boilerplate, and basic features, where most respondents reported gains matching or exceeding 35%.
  • AI-generated code is not a uniform category: small snippets are comparable to human code in cohesion and coupling, so teams can adopt them without significant architectural erosion.
  • Large AI-generated blocks and AI-driven changes to established code bases carry real costs, including lower maintainability, more frequent breaking of existing code, and longer processing times, so they need human architectural review.
  • Since 21% of respondents saw no productivity improvement, AI benefits are not automatic; prompt skill and problem decomposition mediate outcomes.
  • The skills that matter shift toward decomposition and integration: architects who break problems into AI-sized pieces and integrate the results are the ones who capture the productivity gains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not claim, but its data suggest, that productivity studies should measure greenfield and maintenance work separately, since AI appears to help the former and hinder the latter.
  • A controlled experiment with matched teams could test whether the self-reported 35% gain survives objective measurement; the paper's own Tetris substudy is a single-engineer data point, so the survey reference value is itself an assumption to verify.
  • The survey's 'no architectural erosion for snippets' finding is about perceived cohesion and coupling, so an automated code-quality analysis on real repositories would be a natural next test.
  • The reported tendency of AI to write tests that pass existing code rather than validate requirements implies that test generation quality, not just code quality, may degrade as problem size grows.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper reports a two-part study of AI tools in software engineering: a single-engineer, six-week Tetris project comparing development with and without AI assistance, and a survey of 40 practitioners who already use AI tools. The authors conclude that AI tools significantly increase engineer productivity (quantified as 'over 35%'), that productivity gains diminish as project complexity grows, and that adopting AI-generated code snippets does not significantly erode software architecture, while large AI-generated blocks are of significantly lower quality. The paper argues that non-users of AI tools will become less competitive and that architects remain necessary for problem decomposition and solution integration.

Significance. If the claims were supported, the paper would provide field evidence on the productivity and architectural effects of AI-assisted development, a topic of active interest. The study has strengths: the GQM structure is systematic, the raw survey results and Tetris code are made publicly available, and the distinction between small snippets and large AI-generated blocks is a useful hypothesis. However, the central quantitative productivity claim is contradicted by the paper's own reported frequencies, and the survey design—a convenience sample of 40 self-selected AI users with self-reported productivity measures and no control group—cannot support the causal and comparative statements made in the abstract and conclusion. The architecture claims are inferred from majority agreement without statistical tests. The paper is therefore best seen as an exploratory descriptive study whose headline conclusions overstate the evidence.

major comments (4)
  1. [Section 4.3(i) and Figure 6] The abstract and introduction state that an 'overwhelming majority' of participants reported a productivity increase 'over 35%', but the paper's own data in Figure 6 and Section 4.3(i) show only 18 of 40 respondents (45%) chose 'higher' or 'much higher' relative to the 35% reference; 14 chose 'around 35%', 7 'lower', and 1 'zero'. Thus 55% of respondents did not report an increase over 35%. The conclusion in Section 5 that 'the increase in productivity of over 35% is a clear and strong sign' is therefore not supported by the presented frequencies. Moreover, no confidence interval, effect size, or significance test is reported anywhere; the word 'significant' is used colloquially rather than statistically. The central quantitative claim of the paper is internally inconsistent with its own data.
  2. [Section 3.2 (M3.1) and Section 4.2 (M2.1)] The survey's productivity scale is anchored to a reference value of 'up to 35% improvement' taken from the authors' own Tetris substudy, which is a single-engineer, six-week project (Section 3.2). Participants were asked to rate their gains relative to that specific value, and the same 35% value is then used in Section 4.3(i) and Section 5 as the threshold for 'over 35%' increases. This makes the quantitative productivity headline partially circular: the instrument is calibrated with the same n=1 measurement that is later used as the benchmark for the claim. The 35% figure is also presented as a finding of the substudy without any uncertainty quantification, so it cannot serve as a reliable anchor for survey interpretation.
  3. [Section 3.3 and Section 5] The introduction and conclusion claim that individuals using AI tools 'significantly outperform those who do not use such tools'. However, the survey (Section 3.3) recruited only professionals who already use AI tools; there is no comparison group of non-users, and no objective productivity measure. The only comparative evidence comes from the Tetris substudy, which involves a single engineer and three versions, without repeated measures or statistical analysis. These claims are therefore beyond what the study design can support.
  4. [Section 4.3(ii) and Figure 10-b] The claim that adopting AI-generated snippets 'does NOT lead to a significant erosion of software architecture' is based on agreement counts: in Figure 10-b, 25 of 40 respondents (62.5%) agree or strongly agree that snippets are of comparable maintenance quality, while 11 are neutral and 4 disagree. This is a majority opinion, but it is not a statistical demonstration of 'no significant negative influence'; the absence of a negative effect cannot be inferred from majority agreement without a hypothesis test or a confidence interval on the proportion. The same issue applies to Figures 10-c and 11-b. The abstract's wording ('no significant negative influences') overstates the evidence.
minor comments (4)
  1. [Section 4.2] The text says '79% of the surveyed participants experience the same, higher or much higher productivity,' but the counts in Figure 6 (1+17+14=32) give 32/40 = 80%, not 79%.
  2. [Section 4.2, Figure 9-c] The text states that 'over 35% of the surveyed participants' consider integration not to be a challenge, but Figure 9-c shows 1 strongly agree and 10 agree (11/40 = 27.5%), not over 35%.
  3. [Section 3.2] There are typos and style issues: 'summirized' should be 'summarized', and 'Inversion a)' appears three times where 'version a)' is meant.
  4. [References] Reference [1] is incomplete: it lists 'Openai and ashley pilipiszyn.' with a URL but no publication year or proper author formatting.

Circularity Check

1 steps flagged · score 4.0 of 10

The 'over 35% productivity increase' finding is anchored to the authors' own single-engineer Tetris substudy and then reported as an independent survey result; the quantitative headline is partly built into the survey instrument.

  1. self definitional [Section 4.2, with Sections 3.2 (Table 2, M3.1) and 3.3 (Table 3, M2.1)]
    "When considering 35% increase in productivity as a reference value (as observed in our test project), the results (Figure 6) show that 79% of the surveyed participants experience the same, higher or much higher productivity when using AI tools."

    The 35% reference is not an external benchmark; it is the authors' own Tetris substudy result (Table 2, M3.1: 'AI improves productivity by up to 35%'). The survey question (Table 3, M2.1) asks participants to rate productivity 'relative to a reference of up to 35% improvement', so the ordinal categories 'higher' and 'much higher' mean, by definition, more than 35%. Section 4.3(i) then reports 'About 45% of the survey participants experienced an increase in productivity of OVER 35%' by counting those two categories (1+17=18 of 40). The quantitative headline is thus a restatement of the answer-scale anchor rather than an independent measurement; the 'over 35%' result is constructed by an instrument that already assumes 35% as the reference.

full rationale

The architecture-quality conclusions (Section 4.3 ii-iv) are not circular: they are direct self-reports about cohesion, coupling, maintainability, and performance, with no fitted parameter recycled into those claims. The productivity result, however, is partially circular because the 35% threshold is imported from the authors' own single-engineer Tetris substudy and then used as the anchor for the survey's productivity scale, making the later 'over 35%' claim a count of categories defined by that anchor. The paper also overstates the data: only 18/40 (45%) reported 'higher' or 'much higher' (Figure 6), yet the Introduction calls this an 'overwhelming majority' and the Conclusion treats 'over 35%' as 'a clear and strong sign' that AI users outperform non-users, a comparison the survey never measured since only AI users were sampled. These overstatements are correctness/validity concerns rather than further circular reductions. No load-bearing self-citation, imported uniqueness theorem, or ansatz-by-citation pattern appears, so the overall circularity is moderate.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

All quantitative conclusions trace back to either the 40-person convenience survey or the single-engineer Tetris substudy. The 35% reference value is an internal, hand-selected benchmark that is reused as evidence, which is the main free parameter. The axioms are the standard assumptions of survey-based software engineering research; the ad hoc assumption is the transfer of the Tetris result to the survey instrument.

free parameters (1)
  • 35% productivity reference value = 35%
    Chosen from the authors' own single-engineer Tetris substudy (Section 3.2) and used as the anchor for survey question M2.1 (Figure 6). The same value is later cited as evidence of productivity gain ('over 35%') in Section 4.3.
assumptions (4)
  • domain assumption Self-reported subjective productivity is a valid proxy for actual productivity.
    Section 4.2: 'we decided to ask for overall subjective feeling of productivity'. All productivity conclusions rely on this unvalidated proxy.
  • domain assumption A convenience sample of 40 practitioners recruited from the authors' network and LinkedIn is representative of AI-using software engineers.
    Section 3.3 describes recruitment; no sampling frame, response rate, or representativeness analysis is provided, yet conclusions are generalized to the industry.
  • ad hoc to paper The 35% productivity improvement from a one-person, six-week Tetris experiment is a meaningful baseline for survey respondents.
    Sections 3.2 and 4.2 use this internal measurement as the reference category in the survey's productivity question, and Section 4.3 treats 'over 35%' as a finding.
  • domain assumption Majority agreement on Likert items can establish the absence of negative effects on software architecture.
    Section 4.3 conclusions (ii) and (iv) infer 'no significant erosion' from agreement distributions, with no statistical test or objective architecture metric.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Impact of AI-Generated Solutions on Software Architecture and Productivity: Results from a Survey Study." pith.science (2026). https://pith.science/paper/AIFEBIVR

@misc{pith2026250617833,
  author       = {Pith},
  title        = {Pith review of: The Impact of AI-Generated Solutions on Software Architecture and Productivity: Results from a Survey Study},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AIFEBIVR}},
  note         = {Machine review of arXiv:2506.17833}
}
read the original abstract

AI-powered software tools are widely used to assist software engineers. However, there is still a need to understand the productivity benefits of such tools for software engineers. In addition to short-term benefits, there is a question of how adopting AI-generated solutions affects the quality of software over time (e.g., maintainability and extendability). To provide some insight on these questions, we conducted a survey among software practitioners who use AI tools. Based on the data collected from our survey, we conclude that AI tools significantly increase the productivity of software engineers. However, the productivity benefits of using AI tools reduce as projects become more complex. The results also show that there are no significant negative influences of adopting AI-generated solutions on software quality, as long as those solutions are limited to smaller code snippets. However, when solving larger and more complex problems, AI tools generate solutions of a lower quality, indicating the need for architects to perform problem decomposition and solution integration.

Figures

Figures reproduced from arXiv: 2506.17833 by the authors.

Figure 1
Figure 1. AI-based and AI-enabled tools in SDLC 2 Related Work There exist several studies that investigate how effective AI tools are for code generation [6] [16] [1]. Engineers also use AI tools for other purposes: program repair [18][14][29], code summarization [2][11], and generating software architecture solutions [12] [21]. In addition, there are several studies that investigate quality of solutions that these tools pro… view at source ↗
Figure 2
Figure 2. Participants by a) institution size, b) experience (PM - Project Manager), c) role, and d) office location [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Usage of AI for Coding Tasks ChatGPT Copilot Claude AI grammarly GO GPT Pilot Cursor IDE Gemini Amazon Q CodeWhisperer Open-source projects In-house tools 0 10 20 30 32 21 5 3 3 3 2 1 1 1 1 Participants [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: AI tools used by the survey participants [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Tool combinations used by the survey participants [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Perceived productivity improvements from AI [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 8
Figure 8. Figure 8: Participant agreement levels: (a) The best way to use AI tools for improving productivity is to break [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Participant agreement levels on AI limitations for code understanding and integration: (a) Using AI [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: Perceived quality of AI-generated code: (a) When generating full applications or large portions of [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]
Figure 11
Figure 11. Figure 11: Participant responses on characteristics of AI-generated code: (a) AI-generated code rarely fulfils [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 17 canonical work pages

  1. [1]

    https://openai.com/blog/gpt-3-apps/, accessed: 2023-1-8

    Openai and ashley pilipiszyn. https://openai.com/blog/gpt-3-apps/, accessed: 2023-1-8

    work page 2023

  2. [2]

    In: Automated Software Engineering

    Ahmed, T., Devanbu, P.: Few-shot training llms for project-specific code-summarization. In: Automated Software Engineering. 2022, NY, USA (2023) AI: Productivity and architecture 13

    work page 2023

  3. [3]

    In: 2010 Fifth International Conference on Software Engineering Advances

    de Barros Sampaio, S.C., Barros, E.A., de Aquino, G.S., e Silva, M.J.C., de Lemos Meira, S.R.: A review of productivity factors and strategies on software development. In: 2010 Fifth International Conference on Software Engineering Advances. pp. 196–204 (2010). https://doi.org/10.1109/ICSEA.2010.37

    work page doi:10.1109/icsea.2010.37 2010

  4. [4]

    IEEE Transactions on Software EngineeringSE-10(6), 728–738 (1984)

    Basili, V.R., Weiss, D.M.: A methodology for collecting valid software engineering data. IEEE Transactions on Software EngineeringSE-10(6), 728–738 (1984). https://doi.org/10.1109/TSE.1984.5010301

    work page arXiv 1984

  5. [5]

    Bencheikh, L., Höglund, N.: Exploring the efficacy of chatgpt in generating requirements: An experimental study (2023)

    work page 2023

  6. [6]

    https://doi.org/https://doi.org/10.1016/j.mlwa.2024.100526, https://www.sciencedirect.com/science/article/pii/S2666827024000021

    Bucaioni, A., Ekedahl, H., Helander, V., Nguyen, P.T.: Programming with chatgpt: How far can we go? Machine Learning with Applications 15, 100526 (2024). https://doi.org/https://doi.org/10.1016/j.mlwa.2024.100526, https://www.sciencedirect.com/science/article/pii/S2666827024000021

    arXiv 2024

  7. [7]

    ACM Trans

    Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., Ye, W., Zhang, Y., Chang, Y., Yu, P.S., Yang, Q., Xie, X.: A survey on evaluation of large language models. ACM Trans. Intell. Syst. Technol.15(3) (Mar 2024). https://doi.org/10.1145/3641289, https://doi.org/10.1145/3641289

    doi:10.1145/3641289 2024

  8. [8]

    CoRRabs/2103.07115 (2021), https://arxiv.org/abs/2103.07115

    Ciniselli, M., Cooper, N., Pascarella, L., Poshyvanyk, D., Penta, M.D., Bavota, G.: An empirical study on the usage of BERT models for code completion. CoRRabs/2103.07115 (2021), https://arxiv.org/abs/2103.07115

    arXiv 2021

Show all 30 references
  1. [9]

    Fan, Z., Gao, X., Mirchev, M., Roychoudhury, A., Tan, S.H.: Automated repair of programs from large language models (2023)

    2023

  2. [10]

    He, J., Treude, C., Lo, D.: Llm-based multi-agent systems for software engineering: Vision and the road ahead (2024), https://arxiv.org/abs/2404.04834

    2024 arXiv

  3. [11]

    Hu, X., Li, G., Xia, X., Lo, D., Jin, Z.: Deep code comment generation with hybrid lexical and syntactical information. Emp. Softw. Engg.25(3), 2179–2217 (2020)

    2020

  4. [12]

    In: 2024 IEEE 21st International Conference on Software Architecture Companion (ICSA-C)

    Jahic, J., Sami, A.: State of Practice: LLMs in Software Engineering and Software Architecture . In: 2024 IEEE 21st International Conference on Software Architecture Companion (ICSA-C). pp. 311–318. IEEE Computer Society, Los Alamitos, CA, USA (Jun 2024). https://doi.org/10.11...

    2024

  5. [13]

    In: 2021 IEEE 18th International Conference on Software Architecture Companion (ICSA-C)

    Jahić, J., Roitsch, R., Grzymkowski, L.: Knowledge-based adequacy assessment approach to support ai adoption. In: 2021 IEEE 18th International Conference on Software Architecture Companion (ICSA-C). pp. 8–14 (2021). https://doi.org/10.1109/ICSA-C52384.2021.00008

    2021

  6. [14]

    In: Proceedings of the 44th International Conference on Software Engineering

    Jain, N., Vaidyanath, S., Iyer, A., Natarajan, N., Parthasarathy, S., Rajamani, S., Sharma, R.: Jigsaw: Large language models meet program synthesis. In: Proceedings of the 44th International Conference on Software Engineering. p. 1219–1231. ICSE ’22, New York, NY, USA (2022)

    2022

  7. [15]

    Kasunic, M.: Designing an effective survey. Tech. rep. (2005)

    2005

  8. [16]

    In: ICSE

    Kim, S., Zhao, J., Tian, Y., Chandra, S.: Code prediction by feeding trees to transformers. In: ICSE. pp. 150–162 (2021)

    2021

  9. [17]

    CoRRabs/2102.02017 (2021), https://arxiv.org/abs/2102.02017

    Mastropaolo, A., Scalabrino, S., Cooper, N., Nader-Palacio, D., Poshyvanyk, D., Oliveto, R., Bavota, G.: Studying the usage of text-to-text transfer transformer to support code-related tasks. CoRRabs/2102.02017 (2021), https://arxiv.org/abs/2102.02017

    2021 arXiv

  10. [18]

    Moradi Dakhel, A., Majdinasab, V., Nikanjam, A., Khomh, F., Desmarais, M.C., Jiang, Z.M.J.: Github copilot ai pair programmer: Asset or liability? Journal of Systems and Software203, 111734 (2023)

    2023

  11. [19]

    Nguyen-Duc,A.,Abrahamsson,P.,Khomh,F.(eds.):GenerativeAIforEffectiveSoftwareDevelopment.Springer, Cham (June 2024)

    2024

  12. [20]

    Pearce, H.A., Tan, B., Ahmad, B., Karri, R., Dolan-Gavitt, B.: Can openai codex and other large language models help us fix security bugs? ArXivabs/2112.02125 (2021)

    2021 arXiv

  13. [21]

    In: SoutheastCon 2025

    Ramachandran, R.: Transforming software architecture design with intelligent assistants-a comparative analysis. In: SoutheastCon 2025. pp. 1446–1454 (2025). https://doi.org/10.1109/SoutheastCon56624.2025.10971683

    2025

  14. [22]

    Automated Software Engineering31(1), 26 (2024)

    Sauvola, J., Tarkoma, S., Klemettinen, M., Riekki, J., Doermann, D.: Future of software development with generative ai. Automated Software Engineering31(1), 26 (2024). https://doi.org/10.1007/s10515-024-00426-z, https://doi.org/10.1007/s10515-024-00426-z

    2024 doi

  15. [23]

    John Wiley and Sons, Ltd (2002)

    van Solingen (Revision), R., Basili (Original article, 1994 ed.), V., Caldiera (Original article, 1994 ed.), G., Rombach (Original article, 1994 ed.), H.D.: Goal Question Metric (GQM) Ap- proach. John Wiley and Sons, Ltd (2002). https://doi.org/https://doi.org/10.1002/04710289...

    2002 doi

  16. [24]

    https://stackoverflow.co/labs/2024-developer-survey- insights-for-ai-ml/ (2024), accessed: 08/09/2024

    StackOverflow: 2024 developer survey insights for ai/ml. https://stackoverflow.co/labs/2024-developer-survey- insights-for-ai-ml/ (2024), accessed: 08/09/2024

    2024

  17. [25]

    Tian, H., Lu, W., Li, T.O., Tang, X., Cheung, S.C., Klein, J., Bissyandé, T.F.: Is chatgpt the ultimate program- ming assistant – how far is it? (2023), https://arxiv.org/abs/2304.11938

    2023 arXiv

  18. [26]

    ACM Trans

    Tufano, M., Watson, C., Bavota, G., Penta, M.D., White, M., Poshyvanyk, D.: An empirical study on learning bug-fixing patches in the wild via neural machine translation. ACM Trans. Softw. Eng. Methodol.28(4) (sep 2019) 14 Amasanti and Jahić

    2019

  19. [27]

    Waseem, M., Das, T., Ahmad, A., Liang, P., Fehmideh, M., Mikkonen, T.: Chatgpt as a software development bot: A project-based study (2024), https://arxiv.org/abs/2310.13648

    2024 arXiv

  20. [28]

    ACM Trans

    Yang, J., Jin, H., Tang, R., Han, X., Feng, Q., Jiang, H., Zhong, S., Yin, B., Hu, X.: Harnessing the power of llms in practice: A survey on chatgpt and beyond. ACM Trans. Knowl. Discov. Data18(6) (Apr 2024). https://doi.org/10.1145/3649506, https://doi.org/10.1145/3649506

    2024 doi

  21. [29]

    Zhang, J., Cambronero, J., Gulwani, S., Le, V., Piskac, R., Soares, G., Verbruggen, G.: Repairing bugs in python assignments using large language models (2022)

    2022

  22. [30]

    Zhong, L., Wang, Z., Shang, J.: Debug like a human: A large language model debugger via verifying runtime execution step-by-step (2024), https://arxiv.org/abs/2402.16906

    2024 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.