From Generation to Adaptation: Comparing AI-Assisted Strategies in High School Programming Education

Songzan Wang; Tong Hu

REVIEW 3 major objections 5 minor 1 cited by

From Generation to Adaptation: Comparing AI-Assisted Strategies in High School Programming Education

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Starting from small working examples instead of blank prompts lets novice student teams finish every AI-assisted project, where generating from scratch finished only one in five.

desk verdict Useful hypothesis-generating case study on MFU-based LCA scaffolding, but the 20% vs 100% contrast is weaker than the abstract claims because the MVP outcome is partly built into the MFU starting material. read the letter →

arxiv 2506.15955 v1 pith:HQJPX5NF submitted 2025-06-19 cs.CY cs.SE

Tong Hu , Songzan Wang This is my paper · ORCID

classification cs.CYcs.SE

keywords AI-assistedprogrammingscaffoldingLLMcodingagentsK-12educationhuman-AIcollaborationMinimalFunctionalUnitsnoviceprogrammersMVPcompletion

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This exploratory case study claims that novice programmers get far better results from an AI coding agent when they adapt small, already-working code examples than when they ask the agent to generate a whole project from an abstract description. In two three-hour sessions with five high school teams building mini-programs for a competition, only one of five teams reached a working minimum viable product under the from-scratch strategy, while all five succeeded when they started from Minimal Functional Units and used structured, multi-step prompts to modify them. The paper argues that the difference is not raw AI capability but instructional design: coding agents are better at pattern modification within rich context than at de novo generation, and the concrete example reduces the cognitive load on students. If the claim holds, classroom practice should shift from prompting for whole applications toward curating small examples and teaching an analyze-then-adapt workflow.

What carries the argument

The central object is the Minimal Functional Unit (MFU): a small, functional application from an official example library that demonstrates a feature relevant to the target project. Its job is to serve as a starting point that is close enough to the student's goal to be understood, but far enough to require creative adaptation—the paper calls this the 'adjacent possible.' The supporting mechanism is structured prompting: instead of one abstract request, students issue multi-step prompts that analyze the MFU, compare it with their requirements, and specify what to keep and remove. Together these two scaffolds carry the argument by converting LCA use from void-to-code generation, where the model must infer entire architectures, to context-rich pattern transformation, where the model's modification ability and the student's comprehension can both succeed.

What would settle it

Run a counterbalanced two-condition study in which the same type of novice students build equivalent mini-program projects, one condition starting from matching MFUs with structured prompts and the other generating from abstract specifications with identical prompting guidance; if from-scratch generation matches or beats MFU adaptation on MVP completion rates, the central claim would fail.

Watch

Extended reading notes

Core claim

The paper's central claim is that an MFU-based '1-to-100' approach—students take a small, functional example relevant to their goal and use the coding agent to adapt it—is markedly more effective than a From-Scratch '0-to-1' approach for novice high school coders who have basic programming knowledge but no project experience. The observed evidence is the jump from 20% (1 of 5) to 100% (5 of 5) MVP completion across two sequential sessions. The authors further claim this happens because the approach aligns with two things at once: it gives the coding agent the context-rich patterns it handles best, and it gives students a concrete anchor that turns an open-ended build task into a bounded adaptation task. The paper packages this as a dual-scaffolding model, combining technical scaffolding (MFUs) with pedagogical scaffolding (structured multi-step prompts), and concludes that successful LCA integration depends more on pedagogy than on model capability.

Load-bearing premise

The paper attributes the Phase 2 improvement to the MFU strategy, but the design changed several things at once—session order, prior practice, instructor intervention, and prompting style—so whether the MFU itself, rather than these co-occurring changes, caused the 20% to 100% jump is not established.

Editorial extensions

If this is right

K-12 programming courses using coding agents should probably supply curated starter examples rather than asking novices for abstract, from-scratch prompts.
Prompt instruction should teach an 'analyze, compare, then modify' sequence that references specific components of the example code.
The success of the MFU approach suggests that a coding agent's classroom value should be judged together with the instructional context, not by generation capability alone.
The sequential 20%-to-100% gap is strong enough to justify controlled experiments isolating the MFU variable from practice and instructor effects.
Teachers' role shifts from delivering content to orchestrating student-agent interaction: process scaffolding can matter more than content delivery.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the two sessions were sequential with no control group, the paper's own data cannot separate the MFU strategy from order, practice, or the instructor's added prompting guidance; a counterbalanced crossover would be needed to pin the cause.
The 'adjacent possible' idea suggests a testable difficulty curve: if MFUs are too trivial the task adds nothing, and if they are too complex they recreate from-scratch failure; an experiment varying MFU size could map that curve.
The same pattern likely applies outside mini-programs—for example, web app frameworks, game engines, or data analysis templates—wherever small examples can be adapted with an agent, though this generalisation is not tested in the paper.
A second untested separation is whether structured multi-step prompting alone, without MFUs, could deliver most of the benefit; the paper treats the two scaffolds as a package.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

What you should know: this is a small, honest exploratory case study (n=5, two 3-hour sessions) comparing LCA code generation from scratch vs adapting minimal functional units (MFUs) for WeChat Mini Programs. The reported pattern is 20% vs 100% MVP completion. The genuinely useful bit is the practical hypothesis: novice programmers and coding agents both do better when the starting point is a small working example to modify rather than a blank page and an abstract spec. That fits a lot of prior work on worked examples and retrieval-augmented generation, and the paper says so.

What it does well: the authors are transparent about the small sample, single site, instructor dual role, and lack of systematic measures. The limitation section (4.3) explicitly says the sequential design "prevents pure causal claims." That is the right instinct. They also name the mechanism—pattern modification is where LCAs are strong—and link it to existing literature. For a practitioner who wants a concrete strategy, this is a reasonable starting point.

The soft spots are real and one is structural. The biggest is that the outcome measure is largely contained in the starting material. Section 2.4 defines an MVP as a functioning Mini Program demonstrating at least one core feature; Section 2.2 defines an MFU as a small, functional application demonstrating core features relevant to project targets. So any team that picks a relevant MFU begins Phase 2 already meeting the MVP definition. Table 2 makes this concrete: the myopia team kept the camera module and dropped classification, so their "MVP" core feature was inherited from the source code. The 100% completion may reflect the initial MFU, not the adaptation process. The sequential design, simultaneous introduction of structured prompting, no control, and instructor field notes as the only data make the causal claim even weaker. The abstract and conclusion still say "demonstrated" and "proved," which overstates what this design can support.

That said, the paper does not want to be read as a controlled experiment; it frames itself as exploratory. The gap is between that framing and the strong verbs in the abstract. The empirical observation itself (1/5 vs 5/5) is about as extreme as you can get with n=5, and the failure modes in Phase 1 (camera activation, cloud access, device deployment) are plausible and useful for educators.

Who this is for: K-12 educators and researchers working on AI-assisted programming pedagogy. It deserves a serious referee: it is short, readable, and proposes a testable framework. A reviewer should push for softening the causal language and ideally a follow-up with counterbalancing and a proper control.

Bottom line: worthwhile as a hypothesis-generating case study, not as evidence for the claim as stated. I'd cite it for the MFU idea and would bring it to a reading group to discuss the outcome-measure overlap. Send it to review, but expect revision.

Referee Report

3 major / 5 minor

Summary. This exploratory case study compares two LCA-assisted programming strategies with five novice high school students preparing for a WeChat Mini Program competition. In Phase 1, students used a From-Scratch approach, generating code from abstract specifications, and only 1 of 5 teams (20%) achieved a functional MVP. In Phase 2, students adapted pre-existing Minimal Functional Units (MFUs) using LCAs, and all 5 teams (100%) achieved MVPs. The paper argues that the MFU-based approach is markedly more effective because it aligns with LCA strengths in pattern modification and provides cognitive scaffolds, and it proposes a dual-scaffolding model combining technical MFU support with structured prompting. The paper is transparent about its small sample, single-site context, and sequential design, and it explicitly disclaims pure causal inference in the limitations section.

Significance. The pedagogical question—whether adapting existing functional code is a better scaffold for novice programmers than generating whole projects from scratch—is timely and practically relevant. If the MFU-based approach genuinely produced the observed improvement, the paper would offer actionable guidance for K-12 educators integrating coding agents into classrooms. The paper also introduces a testable dual-scaffolding framework and provides a detailed, honest account of a classroom intervention. However, the evidence presented does not support the causal claim as stated. The overlap between the MVP outcome definition and the MFU starting material creates a structural confound, and the sequential without-control design prevents attribution of the improvement to the MFU strategy. The paper's value lies more in generating hypotheses and heuristic guidance than in demonstrating effectiveness.

major comments (3)

[§2.2, §2.4, Table 2] The outcome measure and the intervention are structurally confounded. An MVP is defined in §2.4 as 'a functioning Mini Program demonstrating at least one core feature,' while an MFU is defined in §2.2 as a 'small, functional application from the official WeChat Mini Program library that demonstrate core features relevant to project targets.' Consequently, any team that selects a relevant MFU has already satisfied the MVP criterion before any adaptation work begins. Table 2 makes this concrete: the Myopia Simulator team kept the camera/video module from the object classifier MFU and removed the classification features, so the core feature of the final MVP was present in the source file before student or LCA adaptation. The 100% Phase 2 completion rate may therefore reflect the initial MFU rather than the MFU-based process, the structured prompting, or the LCA's pattern-modification strengths. This undermines the central claim in §1 and the abstract that the MFU approach 'proved markedly more effective.'
[§2.3, §3.2, §4.3] The causal attribution is not supported by the design. Phase 2 always followed Phase 1, with no counterbalancing or control condition, and the instructor introduced both MFUs and structured prompting simultaneously (§3.2). Thus the improvement from 20% to 100% could be due to order, practice, the simultaneous addition of structured prompting, or increased instructor familiarity, rather than to the MFU strategy itself. The paper acknowledges in §4.3 that 'the sequential design prevents pure causal claims,' yet the abstract and conclusion still use verbs such as 'demonstrating' and 'proved.' The claims in the abstract and §1 should be reworded to reflect the exploratory, non-causal nature of the evidence.
[§3.2, §4.1] The explanatory account of why MFUs work is validated using the same outcome data that motivated the intervention. The instructor observed that the LCA performed better when modifying existing code, then introduced MFUs and structured prompting, and then used the improved Phase 2 outcomes to confirm the explanation. This interpretive circularity weakens the claims in RQ2 and RQ3 about the underlying factors. Independent evidence—such as systematic student trace data, cognitive load measures, or structured student reflections—would be needed to support the proposed mechanism, but the paper's data consist primarily of instructor field notes.

minor comments (5)

[Abstract] The phrase 'demonstrating that effective LCA integration depends less on AI capabilities than on instructional design' overstates the evidence; given the study's design, a word such as 'suggesting' would be more accurate.
[§1] There is a typo in the first sentence: 'increasing recognized' should be 'increasingly recognized.'
[References] Reference [5] contains a likely typo in the author name 'Kal-yanjabi' and lacks complete publication venue information; please verify and complete the citation.
[§1] The introduction refers to 'Chapter 2' while the rest of the paper uses 'Section'; please use consistent terminology.
[Table 1] In Table 1, the 'Outcome' column uses 'MVP Achieved' for one row but 'Stalled' for others; capitalizing 'MVP achieved' or using a consistent format across rows would improve readability.

Circularity Check

1 steps flagged · score 6.0 of 10

Phase 2's 100% MVP completion is partly guaranteed by the paper's own definitions: an MFU is already a functional application demonstrating core features, and an MVP is defined as a functioning application demonstrating at least one core feature.

self definitional [Section 1 (MFU definition) and Section 2.4 (MVP definition); applied in Table 2]
"MFUs—defined for this study as small, functional applications that demonstrate core features relevant to target projects. ... An MVP was defined as a functioning Mini Program demonstrating at least one core feature."

The central claim is that the MFU-based approach 'proved markedly more effective' at producing functional projects, but the outcome measure (MVP = functioning Mini Program demonstrating at least one core feature) is entailed by the definition of the intervention input (MFU = small, functional application demonstrating core features relevant to target projects). A relevant MFU therefore already meets the MVP criterion, or contains the exact core feature that stalled Phase 1. Table 2 confirms this: the Myopia Simulator team chose an object-classifier MFU with a working camera/video module—the exact bottleneck from Table 1—and kept that module.

full rationale

The paper is an empirical case study, not a derivational model, so most circularity patterns (fitted parameters, self-citation uniqueness theorems, ansatz-smuggled citations) do not apply. The only shared-author citation (reference [2], if 'Wang, S.' is the current co-author) is background material and is not load-bearing. The qualitative observations—fewer debugging requests, more structured prompts, greater confidence—are independent of the definitional issue and give the paper real content. However, the headline quantitative result (20% vs 100% MVP completion) is partially forced by the paper's own definitions: an MFU is already a functional application demonstrating core features, and an MVP is defined as a functioning application demonstrating at least one core feature. Because the central claim leans on that completion-rate contrast, the circularity score is 6 rather than 0-2. The paper's own limitation statement, 'the sequential design prevents pure causal claims,' tempers but does not repair the definitional overlap, and the abstract/conclusion nevertheless assert that the MFU approach 'proved markedly more effective.'

Assumptions & free parameters 0 free parameters · 4 assumptions · 2 invented entities

No quantitative model or fitted parameters; the ledger lists the implicit assumptions of the causal interpretation. The central claim depends on the sequential design assumption, a domain assumption about LCA behavior, a measurement assumption about field notes, and a background assumption about participant skills.

assumptions (4)

ad hoc to paper The sequential design without counterbalancing or control allows causal attribution of the Phase 2 improvement to the MFU-based strategy.
Section 2.3 and Section 3.2 present the pedagogical shift after Phase 1; this assumption underlies the conclusion that MFU adaptation, rather than practice or order or instructor scaffolding changes, caused the 100% completion rate.
domain assumption LLM coding agents perform better with relevant code context than with de novo generation.
Invoked in Section 4.1 via reference [5]; it bridges the observed outcome to the mechanism, but is not independently tested in this study.
domain assumption Instructor field notes accurately capture student behaviors, technical challenges, and MVP completion.
Section 2.4 states data consisted primarily of instructor field notes; the dual researcher-teacher role is acknowledged as potential bias.
domain assumption Students' AP CS A background is sufficient to benefit from MFU adaptation without additional programming instruction.
Section 2.1 describes participants as having completed AP CS A but lacking WeChat Mini Program experience; the study assumes this baseline makes the MFU scaffold effective.

invented entities (2)

Minimal Functional Unit (MFU) as a named pedagogical construct
purpose: Small, functional code examples from the official WeChat Mini Program library used as starting points for LCA-assisted adaptation.
The code examples are publicly available, but the specific MFU framing and its claimed scaffolding benefit are defined and validated within this study only.
Dual-scaffolding model (MFUs plus structured prompting)
purpose: Explains why Phase 2 succeeded by combining technical scaffolds (MFUs) with pedagogical guidance (structured prompting strategies).
Proposed post hoc from the same two-session case; no external validation or falsifiable handle outside this paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of From Generation to Adaptation: Comparing AI-Assisted Strategies in High School Programming Education." pith.science (2026). https://pith.science/paper/HQJPX5NF

@misc{pith2026250615955,
  author       = {Pith},
  title        = {Pith review of: From Generation to Adaptation: Comparing AI-Assisted Strategies in High School Programming Education},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HQJPX5NF}},
  note         = {Machine review of arXiv:2506.15955}
}

read the original abstract

This exploratory case study investigated two contrasting pedagogical approaches for LCA-assisted programming with five novice high school students preparing for a WeChat Mini Program competition. In Phase 1, students used LCAs to generate code from abstract specifications (From-Scratch approach), achieving only 20% MVP completion. In Phase 2, students adapted existing Minimal Functional Units (MFUs), small, functional code examples, using LCAs, achieving 100% MVP completion. Analysis revealed that the MFU-based approach succeeded by aligning with LCA strengths in pattern modification rather than de novo generation, while providing cognitive scaffolds that enabled students to navigate complex development tasks. The study introduces a dual-scaffolding model combining technical support (MFUs) with pedagogical guidance (structured prompting strategies), demonstrating that effective LCA integration depends less on AI capabilities than on instructional design. These findings offer practical guidance for educators seeking to transform AI tools from sources of frustration into productive learning partners in programming education.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Evaluating the Effectiveness of Large Language Models in Solving Simple Programming Tasks: A User-Centered Study
cs.HC 2025-07 reject novelty 5.0 of 10

A within-subjects experiment with 15 high school students reports faster task completion with a collaborative ChatGPT-4o style than with a passive style, but the result is not significant versus proactive style and is...

Reference graph

Works this paper leans on

15 extracted references · 9 canonical work pages · cited by 1 Pith paper

[1]

A., Denny, P., Finnie-Ansley, J., Luxton-Reilly, A., Prather, J., and Santos, E

Becker, B. A., Denny, P., Finnie-Ansley, J., Luxton-Reilly, A., Prather, J., and Santos, E. A. (2022). Programming Is Hard – Or at Least It Used to Be: Educational Opportunities And Challenges of AI Code Generation. arXiv:2212.01020 [cs]

arXiv 2022
[2]

S., and Wen, Q

Chu, Z., Wang, S., Xie, J., Zhu, T., Yan, Y., Ye, J., Zhong, A., Hu, X., Liang, J., Yu, P. S., and Wen, Q. (2025). LLM Agents for Education: Advances and Applications. arXiv:2503.11733 [cs]

arXiv 2025
[3]

Gardella, N., Pettit, R., and Riggs, S. L. (2024). Performance, Work- load, Emotion, and Self-Efficacy of Novice Programmers Using AI Code Generation. In Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1 , ITiCSE 2024, pages 290–296, New York, NY, USA. Association for Computing Machinery

work page 2024
[4]

M., and Aleven, V

Holstein, K., McLaren, B. M., and Aleven, V. (2019). Co-Designing a Real-Time Classroom Orchestration Tool to Support Teacher–AI Comple- mentarity. Journal of Learning Analytics , 6(2):27–52. Number: 2

work page 2019
[5]

and Papangelis, A

Kal-yanjabi, N. and Papangelis, A. (2025). Deeper insights into retrieval- augmented generation: the role of sufficient context

work page 2025
[6]

Kazemitabaar, M., Chow, J., Ma, C. K. T., Ericson, B. J., Weintrop, D., and Grossman, T. (2023). Studying the effect of AI Code Generators on Supporting Novice Learners in Introductory Programming. In Proceedings 9 of the 2023 CHI Conference on Human Factors in Computing Systems , CHI ’23, pages 1–23, New York, NY, USA. Association for Computing Machinery

work page 2023
[7]

Kim, J. (2024). Types of teacher-AI collaboration in K-12 classroom instruction: Chinese teachers’ perspective. Education and Information Technologies, 29(13):17433–17465

work page 2024
[8]

Li, H., Xu, T., Zhang, C., Chen, E., Liang, J., Fan, X., Li, H., Tang, J., and Wen, Q. (2024). Bringing Generative AI to Adaptive Learning in Education. arXiv:2402.14601 [cs]

arXiv 2024

Show all 15 references

[9]

From idea to app

Lovable (2025). From idea to app. https://docs.lovable.dev/ user-guides/from-idea-to-app . Accessed: 2025-06-18

2025
[10]

Matsuda, N., Weng, W., and Wall, N. (2020). The Effect of Metacogni- tive Scaffolding for Learning by Teaching a Teachable Agent.International Journal of Artificial Intelligence in Education , 30(1):1–37

2020
[11]

B., Hecker, D., Houben, S., Mock, M., Rosenzweig, J., Sicking, J., Schulz, E., Voss, A., and Wrobel, S

Poretschkin, M., Schmitz, A., Akila, M., Adilova, L., Becker, D., Cre- mers, A. B., Hecker, D., Houben, S., Mock, M., Rosenzweig, J., Sicking, J., Schulz, E., Voss, A., and Wrobel, S. (2023). Guideline for Trustworthy Artificial Intelligence – AI Assessment Catalog. arXiv:2307...

2023
[12]

Tan, X., Cheng, G., and Ling, M. H. (2025). Artificial intelligence in teaching and teacher professional development: A systematic review. Computers and Education: Artificial Intelligence , 8:100355

2025
[13]

Wang, F., Zhou, X., Li, K., Cheung, A. C. K., and and, M. T. (2025). The effects of artificial intelligence-based interactive scaffolding on sec- ondary students’ speaking performance, goal setting, self-evaluation, and motivation in informal digital learning of english. Inter...

2025
[14]

White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., and Schmidt, D. C. (2023). A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT. arXiv:2302.11382 [cs] version: 1. 10

2023 arXiv
[15]

S., Jong, M

Zhai, X., Chu, X., Chai, C. S., Jong, M. S. Y., Istenic, A., Spector, M., Liu, J.-B., Yuan, J., and Li, Y. (2021). A Review of Artificial Intelli- gence (AI) in Education from 2010 to 2020. Complexity, 2021(1):8812542. eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1155/20...

2021 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

A., Denny, P., Finnie-Ansley, J., Luxton-Reilly, A., Prather, J., and Santos, E

Becker, B. A., Denny, P., Finnie-Ansley, J., Luxton-Reilly, A., Prather, J., and Santos, E. A. (2022). Programming Is Hard – Or at Least It Used to Be: Educational Opportunities And Challenges of AI Code Generation. arXiv:2212.01020 [cs]

arXiv 2022

[2] [2]

S., and Wen, Q

Chu, Z., Wang, S., Xie, J., Zhu, T., Yan, Y., Ye, J., Zhong, A., Hu, X., Liang, J., Yu, P. S., and Wen, Q. (2025). LLM Agents for Education: Advances and Applications. arXiv:2503.11733 [cs]

arXiv 2025

[3] [3]

Gardella, N., Pettit, R., and Riggs, S. L. (2024). Performance, Work- load, Emotion, and Self-Efficacy of Novice Programmers Using AI Code Generation. In Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1 , ITiCSE 2024, pages 290–296, New York, NY, USA. Association for Computing Machinery

work page 2024

[4] [4]

M., and Aleven, V

Holstein, K., McLaren, B. M., and Aleven, V. (2019). Co-Designing a Real-Time Classroom Orchestration Tool to Support Teacher–AI Comple- mentarity. Journal of Learning Analytics , 6(2):27–52. Number: 2

work page 2019

[5] [5]

and Papangelis, A

Kal-yanjabi, N. and Papangelis, A. (2025). Deeper insights into retrieval- augmented generation: the role of sufficient context

work page 2025

[6] [6]

Kazemitabaar, M., Chow, J., Ma, C. K. T., Ericson, B. J., Weintrop, D., and Grossman, T. (2023). Studying the effect of AI Code Generators on Supporting Novice Learners in Introductory Programming. In Proceedings 9 of the 2023 CHI Conference on Human Factors in Computing Systems , CHI ’23, pages 1–23, New York, NY, USA. Association for Computing Machinery

work page 2023

[7] [7]

Kim, J. (2024). Types of teacher-AI collaboration in K-12 classroom instruction: Chinese teachers’ perspective. Education and Information Technologies, 29(13):17433–17465

work page 2024

[8] [8]

Li, H., Xu, T., Zhang, C., Chen, E., Liang, J., Fan, X., Li, H., Tang, J., and Wen, Q. (2024). Bringing Generative AI to Adaptive Learning in Education. arXiv:2402.14601 [cs]

arXiv 2024