Pith. sign in

REVIEW 2 major objections 37 references

Students guide AI code tools mainly with plain-language “what” comments and spend more effort checking generated code than rewriting those comments.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 10:03 UTC pith:CVB7K42I

load-bearing objection Solid multi-year CS-ed baseline on how students write Copilot comments; the taxonomy and verification-over-rewrite finding are useful, even if the LLM labels rest on a thin validation sample. the 2 major comments →

arxiv 2607.10674 v1 pith:CVB7K42I submitted 2026-07-12 cs.SE cs.AIcs.CY

Commenting with Copilot: A Taxonomy and Multi-Year Analysis of Student Code-Generation Specifications

Nasser Giacaman , Valerio Terragni , Paul Denny , Viraj Kumar This is my paper
classification cs.SE cs.AIcs.CY
keywords AI-assisted programmingcode generationstudent commentsGitHub Copilotcode comprehensionprompt engineeringcomputing education
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

As AI coding tools enter programming classes, students more often describe intended behavior in natural language and let the tool produce code, so specification and verification matter as much as typing syntax. This paper studies four years of undergraduate submissions in which students wrote comments inside starter files to steer an AI code generator, then refined solutions against automated tests. It introduces a three-part taxonomy of those comments—by purpose (what, how, or why), how code-like they sound, and which programming construct they target—and shows that students overwhelmingly wrote natural-language “what” comments, shifted toward procedural “how” comments for multi-step and looping logic, and revised comments far less often than they verified the generated output. The work matters because it treats comment-based AI programming as a combined skill of understanding behavior, expressing intent, and judging correctness, not merely as a faster way to obtain code.

Core claim

Across four yearly cohorts of an object-oriented programming course, students directing an AI code generator with comments mostly wrote natural-language “What” comments that state intended outcomes (about 77%), while multi-step blocks and loop constructs drew a clear shift toward “How” comments that spell out procedures. Comment rewriting across attempts was rare; unchanged and newly added comments dominated, and light, heavy, or full rewrites together accounted for only a few percent of early-attempt comments, with student reflections pointing instead to checking and interpreting generated code as the main iterative work.

What carries the argument

A three-dimensional taxonomy of student code-generation comments: comment type (What / How / Why), code expression level (Natural-Language / Code-Like / Strict-Code), and targeted code construct. Automated labeling of tens of thousands of extracted comments under this taxonomy produces the construct-level distributions and modification patterns that carry the results.

Load-bearing premise

The large-scale percentages and construct-level shifts rest on automated taxonomy labels being accurate enough for the full comment set, with only a small human agreement sample used as a check.

What would settle it

A large, fully human-coded sample (or dual independent labeling with adjudication) that reassigns multi-step or loop comments away from “How,” or that records much higher rewrite rates once IDE-only edits before submission are included, would overturn the main distributional claims.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Comment-based AI exercises can be framed as practice in expressing and verifying behavior rather than as pure code-production drills.
  • Instruction and assessment can treat verification of AI output and test-driven checking as central learning goals, because that is where student effort already concentrates.
  • Teaching materials can anticipate heavier procedural (“how”) guidance for multi-step and iterative constructs and lighter outcome-only comments for simpler ones.
  • If deliberate prompt refinement is a learning goal, it must be scaffolded, because spontaneous comment rewriting is uncommon under this workflow.
  • Educators can design hybrid tasks that keep students responsible for judging correctness rather than treating the tool as an unquestioned answer source.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The what-to-how shift on procedural constructs may generalize beyond comment interfaces to open chat prompting, reflecting a novice default of stating outcomes first and procedures when the logic grows complex.
  • Because rewriting was rare while verification was heavy, unconstrained IDE editing of generated code (disallowed here) is likely to become the default repair path and may undercut practice in clear specification.
  • Pairing comment-based generation with short “explain the failure” reflections could convert verification burden into visible learning rather than invisible friction.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 0 minor

Summary. This paper analyzes four years of student comments written as specifications for GitHub Copilot in a second-year Java OOP course. Students reproduced short classes from screenshots by writing comments (not code), submitted generated solutions to automated tests, and could revise comments across attempts. The authors introduce a three-dimensional taxonomy (comment type: What/How/Why; expression level: Natural-Language/Code-Like/Strict-Code; code construct) and apply an LLM classifier to 52,172 unique comments from 1,161 students and 10,257 attempts. They report that comments are predominantly natural-language What comments (77%), shift toward How for Multi-Step-Block (94.3%) and Loop-Iteration (57.3%), and that students rewrite comments infrequently (Light/Heavy/Full Rewrite ~4.4% of early attempts), focusing instead on verifying generated code. Reflections are thematically analyzed into 13 themes emphasizing speed, verification burden, and prompt precision.

Significance. The work addresses a timely gap: how novices actually write comment-based specifications when AI tools generate code. Strengths include a large multi-year authentic dataset, quantified test-suite adequacy (coverage and mutation scores), blinded human agreement checks on the LLM labels (92.5–94% agreement, κ 0.77–0.93 on n=200), and transparent modification thresholds based on Levenshtein similarity. The taxonomy and the construct-conditioned What→How shift are useful contributions for computing education research on AI-assisted programming, and the finding that verification dominates rewriting has clear pedagogical implications. If the aggregate distributions hold under stronger validation, the paper provides a solid empirical baseline for designing comment-based AI activities.

major comments (2)
  1. §3.3 and Table 1: The central quantitative claims (77% What overall; 94.3% How on Multi-Step-Block; 57.3% How on Loop-Iteration) rest entirely on an LLM pipeline validated only on a single random sample of 200 comments. No confusion matrices, no stratified or per-construct error rates, and no adjudication of the residual Multi-Step-Block category are provided. Because Multi-Step-Block is defined as “multiple actions… no one category dominates” and is itself LLM-assigned, systematic co-assignment of multi-line procedural text to both Multi-Step-Block and How would inflate the headline shift. A modest stratified re-labeling (or at least error rates on the key procedural constructs) is needed before those percentages can be treated as reliable.
  2. §3.2.3 and Table 1: Multi-Step-Block accounts for 16.3% of comments and drives the strongest How result, yet it is a study-specific residual aggregation rather than a standard language construct. The paper should clarify decision rules for when a comment is Multi-Step-Block versus a dominant single construct, and report inter-rater or LLM–human agreement specifically for this category; otherwise the construct-level shift claim is hard to interpret or replicate.

Circularity Check

0 steps flagged

No circularity: observational taxonomy study with independent category definitions and human-validated LLM labels; no predictions reduce by construction.

full rationale

This paper is a multi-year empirical analysis of student comments written to guide GitHub Copilot. It defines a three-dimensional taxonomy (comment type What/How/Why adapted from prior comment-purpose literature; expression level; code construct), applies an LLM classifier validated against a blinded human sample of 200 comments (agreements 92.5–94 %, κ 0.77–0.93), and reports descriptive distributions (Table 1), modification rates across attempts (Figure 1), and reflection themes. The taxonomy categories are stipulated independently of the outcome counts; the classifier is not tuned to force the reported What/How ratios or construct-level shifts; reflection codes are derived from student text rather than reverse-engineered from the quantitative claims. There are no equations, fitted parameters re-labeled as predictions, uniqueness theorems imported from the authors, or self-citation chains that force the central results. Self-citations appear only as ordinary related-work context and do not underwrite the load-bearing empirical claims. Consequently the derivation chain contains no circular steps of the kinds enumerated.

Axiom & Free-Parameter Ledger

1 free parameters · 4 axioms · 2 invented entities

Empirical education paper; load-bearing premises are the reliability of the automated labeling pipeline, the representativeness of the constrained reproduction task, and the Levenshtein cut-offs that define modification categories. No physical constants or free parameters of a physical model appear. The taxonomy categories themselves are invented analytic entities whose utility is demonstrated only inside this study.

free parameters (1)
  • Levenshtein similarity thresholds for modification categories = 0.8 / 0.2
    Light Modification ≥ 0.8, Full Rewrite < 0.2, Heavy Modification in between; thresholds chosen after inspection of representative pairs rather than derived from first principles or external standard.
axioms (4)
  • domain assumption GPT-5.4-family medium-reasoning labels for comment type, expression level and code construct are accurate enough that aggregate percentages and construct-level shifts can be trusted.
    Supported by a 200-comment human agreement check (92.5–94 %, κ 0.77–0.93) but not by full dual coding of the 52 k set; invoked throughout §3.3 and §4.
  • domain assumption Comments extracted from CodeRunner submissions (including carry-forward of unchanged comments) faithfully represent the student–Copilot interaction that occurred in the IDE.
    IDE-only revisions overwritten before submission are invisible; stated as a limitation in §5.
  • domain assumption Passing the provided test suites (high line/branch coverage and mutation scores) is a sufficient proxy for behavioral correctness of the generated code.
    Authors quantify coverage and mutation but note that no suite is exhaustive; used to define successful attempts.
  • domain assumption Standard What/How/Why distinction from software-comment literature transfers to student prompts written for an LLM.
    Adapted from Pascarella et al. and Zhai et al.; §3.2.1.
invented entities (2)
  • Three-dimensional student-comment taxonomy (type × expression level × construct) no independent evidence
    purpose: To classify every extracted comment so that distributions and shifts across constructs and attempts can be quantified.
    Categories are defined by the authors for this study; independent evidence is limited to the internal reliability sample and face validity of examples.
  • Multi-Step-Block construct category no independent evidence
    purpose: Catch-all when multiple actions are covered and no single construct dominates; turns out to be the dominant How hotspot.
    Study-specific aggregation not present in standard Java AST taxonomies; §3.2.3.

pith-pipeline@v1.1.0-grok45 · 17592 in / 3100 out tokens · 33333 ms · 2026-07-14T10:03:04.904767+00:00 · methodology

0 comments
read the original abstract

As AI code tools become integrated into programming environments, students increasingly describe intended behavior in natural language and rely on these tools to generate code, shifting emphasis from code writing to specification. Yet little is known about the comments students write as specifications in AI-assisted programming tasks. We analyze a four-year dataset of undergraduate programming submissions and reflections from tasks in which students wrote comments to guide code generation and refined solutions using test-case feedback. We introduce a taxonomy spanning three dimensions: comment type, code expression level, and code construct. Using automated classification, we examine how these dimensions vary across attempts and how students describe the process in their reflections. Our findings show that students mostly wrote natural-language What comments, shifted toward How comments for more procedural constructs, and focused more on verifying generated code than on repeatedly rewriting comments.

Figures

Figures reproduced from arXiv: 2607.10674 by Nasser Giacaman, Paul Denny, Valerio Terragni, Viraj Kumar.

Figure 1
Figure 1. Figure 1: Pooled comment modification categories across all three exercises and four yearly cohorts for the first 10 attempts of [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Changed comments from question to construct to modification category. The top seven constructs are shown (5,692 of [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 7 canonical work pages

  1. [1]

    Andrews, Lionel C

    James H. Andrews, Lionel C. Briand, and Yvan Labiche. 2005. Is Mutation an Appropriate Tool for Testing Experiments?. InProceedings of the 27th International Conference on Software Engineering. 402–411. doi:10.1145/1062455.1062530

    doi:10.1145/1062455.1062530 2005

  2. [2]

    Claus Brabrand and Paul Denny. 2026. Constructive Alignment in the Age of AI. osf.io/preprints/edarxiv/m9yfk_v1

    2026

  3. [3]

    Robert Chew, John Bollenbacher, Michael Wenger, Jessica Speer, and Annice Kim

  4. [4]

    arXiv:2306.14924 [cs.CL] https://arxiv.org/abs/2306.14924

    LLM-Assisted Content Analysis: Using Large Language Models to Support Deductive Coding. arXiv:2306.14924 [cs.CL] https://arxiv.org/abs/2306.14924

    Pith/arXiv arXiv

  5. [5]

    Henry Coles, Thomas Laurent, Christopher Henard, Mike Papadakis, and An- thony Ventresque. 2016. PIT: a practical mutation testing tool for Java. InPro- ceedings of the 25th International Symposium on Software Testing and Analysis

    2016

  6. [6]

    Shih-Chieh Dai, Aiping Xiong, and Lun-Wei Ku. 2023. LLM-in-the-loop: Leverag- ing Large Language Model for Thematic Analysis. InFindings of the Association for Computational Linguistics: EMNLP 2023, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). ACL, Singapore. doi:10.18653/v1/2023.findings-emnlp.669

    doi:10.18653/v1/2023.findings-emnlp.669 2023

  7. [7]

    Margulieux, and Briana B

    Adrienne Decker, Lauren E. Margulieux, and Briana B. Morrison. 2019. Using the SOLO Taxonomy to Understand Subgoal Labels Effect in CS1. InProceedings of the 2019 ACM Conference on International Computing Education Research(Toronto ON, Canada)(ICER ’19). Association for Computing Machinery, New York, NY, USA, 209–217. doi:10.1145/3291279.3339405

    work page doi:10.1145/3291279.3339405 2019

  8. [8]

    Paul Denny, Viraj Kumar, and Nasser Giacaman. 2023. Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Lan- guage. InProceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1(Toronto ON, Canada)(SIGCSE 2023). Association for Computing Machinery, New York, NY, USA, 1136–1142. doi:10.1145/35...

    doi:10.1145/3545945.3569823 2023

  9. [9]

    Becker, and Brent N

    Paul Denny, Juho Leinonen, James Prather, Andrew Luxton-Reilly, Thezyrie Amarouche, Brett A. Becker, and Brent N. Reeves. 2024. Prompt Problems: A New Programming Exercise for the Generative AI Era. InProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1(Portland, OR, USA)(SIGCSE 2024). Association for Computing Machinery, New...

    doi:10.1145/3626252.3630909 2024

  10. [10]

    Smith, Max Fowler, James Prather, Brett A

    Paul Denny, David H. Smith, Max Fowler, James Prather, Brett A. Becker, and Juho Leinonen. 2024. Explaining Code with a Purpose: An Integrated Approach for Developing Code Comprehension and Prompting Skills. InProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1(Milan, Italy)(ITiCSE 2024). Association for Computing Machi...

    doi:10.1145/3649217.3653587 2024

  11. [11]

    Max Fowler, Binglin Chen, and Craig Zilles. 2021. How should we ‘Explain in plain English’? Voices from the Community. InProceedings of the 17th ACM Conference on International Computing Education Research(Virtual Event, USA) (ICER 2021). Association for Computing Machinery, New York, NY, USA, 69–80. doi:10.1145/3446871.3469738

    work page doi:10.1145/3446871.3469738 2021

  12. [12]

    Hoffmann, Brock Janiczak, Evgeny Mandrikov, and Mirko Friedenhagen

    Marc R. Hoffmann, Brock Janiczak, Evgeny Mandrikov, and Mirko Friedenhagen. [n. d.]. JaCoCo Java Code Coverage Library. https://www.jacoco.org/. Accessed: 2026-05-08

    2026

  13. [13]

    Smith IV, Viraj Kumar, and Paul Denny

    David H. Smith IV, Viraj Kumar, and Paul Denny. 2024. Explain in Plain Language Questions with Indic Languages: Drawbacks, Affordances, and Opportunities. arXiv:2409.20297 [cs.CY] https://arxiv.org/abs/2409.20297

    Pith/arXiv arXiv 2024

  14. [14]

    Ellen Jiang, Edwin Toh, Alejandra Molina, Kristen Olson, Claire Kayacik, Aaron Donsbach, Carrie J Cai, and Michael Terry. 2022. Discovering the Syntax and Strategies of Natural Language Programming with Generative Language Models. InProceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA)(CHI ’22). Association fo...

    doi:10.1145/3491102.3501870 2022

  15. [15]

    Hyoungwook Jin and Juho Kim. 2024. CodeTree: A System for Learnersourcing Subgoal Hierarchies in Code Examples.Proc. ACM Hum.-Comput. Interact.8, CSCW1, Article 31 (April 2024), 37 pages. doi:10.1145/3637308

    work page doi:10.1145/3637308 2024

  16. [16]

    Ernst, Reid Holmes, and Gordon Fraser

    René Just, Darioush Jalali, Laura Inozemtseva, Michael D. Ernst, Reid Holmes, and Gordon Fraser. 2014. Are Mutants a Valid Substitute for Real Faults in Soft- ware Testing?. InProceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE 2014). Association for Computing Machinery, 654–665. doi:10.1145/2635868.2635929

    doi:10.1145/2635868.2635929 2014

  17. [17]

    Majeed Kazemitabaar, Xinying Hou, Austin Henley, Barbara Jane Ericson, David Weintrop, and Tovi Grossman. 2024. How Novices Use LLM-based Code Gen- erators to Solve CS1 Coding Tasks in a Self-Paced Learning Environment. In Proceedings of the 23rd Koli Calling International Conference on Computing Ed- ucation Research(Koli, Finland)(Koli Calling ’23). Asso...

    doi:10.1145/3631802.3631806 2024

  18. [18]

    David Kerschbaumer, Christoph Schatz, Thorsten Ruprechter, Christian Gütl, and Alexander Steinmaurer. 2025. Do Comments Matter? Investigating Students’ Source Code Comment Behaviour and Its Relation to Academic Success in a CS1 Course. InFutureproofing Engineering Education for Global Responsibility, Michael E. Auer and Tiia Rüütmann (Eds.). Springer Natu...

    2025

  19. [19]

    Richard Lobb and Jenny Harlow. 2016. Coderunner: a tool for assessing computer programming skills.ACM Inroads7, 1 (Feb. 2016), 47–51. doi:10.1145/2810041

    work page doi:10.1145/2810041 2016

  20. [20]

    Laurie Murphy, Renée McCauley, and Sue Fitzgerald. 2012. ’Explain in plain English’ questions: implications for teaching. InProceedings of the 43rd ACM Technical Symposium on Computer Science Education(Raleigh, North Carolina, USA)(SIGCSE ’12). ACM, 385–390. doi:10.1145/2157136.2157249

    doi:10.1145/2157136.2157249 2012

  21. [21]

    Sydney Nguyen, Hannah McLean Babe, Yangtian Zi, Arjun Guha, Carolyn Jane Anderson, and Molly Q Feldman. 2024. How Beginning Programmers and Code LLMs (Mis)read Each Other. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems(Honolulu, HI, USA)(CHI ’24). ACM, New York, NY, USA, Article 651, 26 pages. doi:10.1145/3613904.3642706

    doi:10.1145/3613904.3642706 2024

  22. [22]

    Victor-Alexandru Padurean, Alkis Gotovos, Ahana Ghosh, Paul Denny, Juho Leinonen, Andrew Luxton-Reilly, James Prather, and Adish Singla. 2026. Inter- leaving Natural Language Prompting with Code Editing for Solving Programming Tasks with Generative AI Models. https://arxiv.org/abs/2509.14088

    Pith/arXiv arXiv 2026

  23. [23]

    Mike Papadakis, Marinos Kintis, Jie Zhang, Yue Jia, Yves Le Traon, and Mark Harman. 2019. Mutation Testing Advances: An Analysis and Survey. InAdvances in Computers, Atif Memon (Ed.). Vol. 112. Elsevier, 275–378. doi:10.1016/bs.adcom. 2018.03.015

    work page doi:10.1016/bs.adcom 2019

  24. [24]

    Luca Pascarella, Magiel Bruntink, and Alberto Bacchelli. 2019. Classifying code comments in Java software systems.Empirical Software Engineering24, 3 (2019)

    2019

  25. [25]

    Mrigank Pawagi and Viraj Kumar. 2024. Probeable Problems for Beginner- level Programming-with-AI Contests. InProceedings of the 2024 ACM Conference on International Computing Education Research - Volume 1(Melbourne, VIC, Australia)(ICER ’24). Association for Computing Machinery, New York, NY, USA, 166–176. doi:10.1145/3632620.3671108

    doi:10.1145/3632620.3671108 2024

  26. [26]

    Reeves, Jaromir Savelka, David H

    James Prather, Juho Leinonen, Natalie Kiesler, Jamie Gorson Benario, Sam Lau, Stephen MacNeil, Narges Norouzi, Simone Opel, Vee Pettit, Leo Porter, Brent N. Reeves, Jaromir Savelka, David H. Smith, Sven Strickroth, and Daniel Zingaro

  27. [27]

    In2024 Working Group Reports on Innovation and Technology in Computer Science Education(Milan, Italy)(ITiCSE 2024)

    Beyond the Hype: A Comprehensive Review of Current Trends in Genera- tive AI Research, Teaching Practices, and Tools. In2024 Working Group Reports on Innovation and Technology in Computer Science Education(Milan, Italy)(ITiCSE 2024). Association for Computing Machinery, New York, NY, USA, 300–338. doi:10.1145/3689187.3709614

    doi:10.1145/3689187.3709614 2024

  28. [28]

    James Prather, Brent N Reeves, Paul Denny, Juho Leinonen, Stephen MacNeil, Andrew Luxton-Reilly, João Orvalho, Amin Alipour, Ali Alfageeh, Thezyrie Amarouche, Bailey Kimmel, Jared Wright, Musa Blake, and Gweneth Barbre

  29. [29]

    InProceedings of the 27th Australasian Computing Education Conference (ACE ’25)

    Breaking the Programming Language Barrier: Multilingual Prompting to Empower Non-Native English Learners. InProceedings of the 27th Australasian Computing Education Conference (ACE ’25). Association for Computing Machin- ery, New York, NY, USA, 74–84. doi:10.1145/3716640.3716649

    doi:10.1145/3716640.3716649

  30. [30]

    Victor-Alexandru Pădurean, Paul Denny, Alkis Gotovos, and Adish Singla. 2025. Prompt Programming: A Platform for Dialogue-based Computational Problem Solving with Generative AI Models. InProceedings of the 30th ACM Conference on Innovation and Technology in Computer Science Education V. 1(Nijmegen, Netherlands)(ITiCSE 2025). Association for Computing Mach...

    doi:10.1145/3724363.3729094 2025

  31. [31]

    Tingrui Qiao, Caroline Walker, Chris Cunningham, and Yun Sing Koh. 2025. Thematic-LM: A LLM-based Multi-agent System for Large-scale Thematic Anal- ysis. InProceedings of the ACM on Web Conference 2025(Sydney NSW, Australia) (WWW ’25). ACM, 649–658. doi:10.1145/3696410.3714595

    doi:10.1145/3696410.3714595 2025

  32. [32]

    Anshul Shah, Thomas Rexin, Elena Tomson, William G Griswold, Leo Porter, and Adalbert Gerald Soosai Raj. 2025. Evolution of Programmers’ Trust in Generative AI Programming Assistants. InProceedings of the 25th Koli Calling International Conference on Computing Education Research (Koli Calling ’25). ACM, New York, NY, USA, Article 12, 11 pages. doi:10.1145...

    work page doi:10.1145/3769994.3770029 2025

  33. [33]

    Ansh Sharma and James R Wallace. 2025. DeTAILS: Deep Thematic Analysis with Iterative LLM Support. InProceedings of the 7th ACM Conference on Conversational User Interfaces (CUI ’25). Association for Computing Machinery, New York, NY, USA, Article 28, 7 pages. doi:10.1145/3719160.3735657

    work page doi:10.1145/3719160.3735657 2025

  34. [34]

    Smith, Paul Denny, and Max Fowler

    David H. Smith, Paul Denny, and Max Fowler. 2024. Prompting for Compre- hension: Exploring the Intersection of Explain in Plain English Questions and Prompt Writing. InProceedings of the Eleventh ACM Conference on Learning @ Scale(Atlanta, GA, USA)(L@S ’24). ACM, 39–50. doi:10.1145/3657604.3662039

    doi:10.1145/3657604.3662039 2024

  35. [35]

    Explain-in-Plain-English

    David H. Smith and Craig Zilles. 2024. Code Generation Based Grading: Eval- uating an Auto-grading Mechanism for "Explain-in-Plain-English" Questions. InProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1(Milan, Italy)(ITiCSE 2024). Association for Computing Machinery, New York, NY, USA, 171–177. doi:10.1145/3649217.3653582

    doi:10.1145/3649217.3653582 2024

  36. [36]

    Valerio Terragni, Annie Vella, Partha Roop, and Kelly Blincoe. 2025. The future of ai-driven software engineering.ACM Transactions on Software Engineering and Methodology34, 5 (2025), 1–20

    2025

  37. [37]

    Juan Zhai, Xiangzhe Xu, Yu Shi, Guanhong Tao, Minxue Pan, Shiqing Ma, Lei Xu, Weifeng Zhang, Lin Tan, and Xiangyu Zhang. 2020. CPC: automatically classifying and propagating natural language comments via program analysis. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineer- ing(Seoul, South Korea)(ICSE ’20). Association for C...

    doi:10.1145/3377811.3380427 2020