pith. machine review for the scientific record. sign in

arxiv: 2605.13776 · v1 · submitted 2026-05-13 · 💻 cs.HC

Recognition: unknown

"Like Taking the Path of Least Resistance": Exploring the Impact of LLM Interaction on the Creative Process of Programming

Run Huang, Souti Chattopadhyay, Zeinabsadat Saghi

Pith reviewed 2026-05-14 17:36 UTC · model grok-4.3

classification 💻 cs.HC
keywords LLMcreativityprogramminghuman-AI collaborationidea generationcreative momentscollaboration modes
0
0 comments X

The pith

LLM assistance shortens idea-generation periods in programming and reduces creative moments while final solutions contain similar numbers of ideas.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper studies how large language models change creative thinking during programming tasks. In experiments with 20 programmers working on the same problems with and without LLM help, participants using the models spent less time generating ideas, resulting in fewer creative moments. The final code solutions showed roughly the same number of ideas in both conditions, even though LLM-assisted code was often more correct. The work also maps four distinct ways programmers collaborate with LLMs to tackle problems. These observations matter because they show a possible trade-off between faster problem-solving and the depth of human creative engagement in coding.

Core claim

Through a within-subject study with 20 programmers, the authors found that LLM-assisted conditions led to significantly shorter idea-generation periods (p=0.0004) and fewer creative moments (p=0.002). Generated solutions in both conditions contained roughly the same number of ideas. Qualitative analysis identified four human-LLM collaboration modes that support different problem-solving strategies. LLMs helped produce more correct code but did not increase the idea count in solutions.

What carries the argument

Idea-generation periods and creative moments tracked across LLM-assisted and unassisted conditions, plus four identified human-LLM collaboration modes.

If this is right

  • LLM tools may reduce the time programmers spend exploring ideas before settling on solutions.
  • Final code solutions maintain comparable idea diversity with or without LLM assistance.
  • Programmers adopt varied collaboration modes with LLMs depending on the task strategy.
  • LLM design should consider ways to preserve creative moments alongside efficiency gains.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Future coding assistants could include prompts that deliberately extend ideation time to support more creative moments.
  • Similar process shifts might appear in other creative fields such as writing or visual design when AI tools are introduced.
  • Testing whether specific prompting styles can restore creative moment counts without slowing down the work would be a direct next step.

Load-bearing premise

That shorter idea-generation periods and fewer creative moments accurately capture reduced creativity in programming, and that the within-subject design controls for individual differences and learning effects.

What would settle it

A replication using think-aloud protocols and independent creativity ratings that finds equal idea-generation durations and creative moments across conditions would falsify the main process findings.

Figures

Figures reproduced from arXiv: 2605.13776 by Run Huang, Souti Chattopadhyay, Zeinabsadat Saghi.

Figure 1
Figure 1. Figure 1: Experimental Design and Analysis. An overview of the randomized study comparing Unassisted versus LLM-Assisted creative [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Participant Study Protocol. After recruitment, participants signed consent forms and familiarized themselves with the coding [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Comparison of all participants (a) total frequency of each actions (b) total time spent on each each task across conditions [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: UMAP projections of solution embeddings overlaid with 2D kernel density estimation (KDE) contours for Algorithmic tasks [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Empirical cumulative distribution functions (ECDFs) of pairwise semantic distance ( [PITH_FULL_IMAGE:figures/full_fig_p015_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Comparison of frequency and duration distributions of each programming stage in both conditions [PITH_FULL_IMAGE:figures/full_fig_p018_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Number of creative moments during each problem solving stage across unassisted and LLM-assisted conditions [PITH_FULL_IMAGE:figures/full_fig_p020_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: The top grid shows individual participants’ collaboration strategies (Human, Primarily Human, Primarily LLM, or Fully LLM) [PITH_FULL_IMAGE:figures/full_fig_p022_8.png] view at source ↗
read the original abstract

Creativity is fundamentally human. As AI takes on more of the generative work that once required human imagination, despite documented limitations in creative ability, a critical question emerges: How does GenAI affect users' creativity? Through a within-subject study followed by retrospective interviews with (N=20) programmers, we investigated the impact of LLMs on participants' process of creative thinking in programming and the creativity of generated solutions. Across two conditions (LLM-assisted vs. unassisted), participants using LLMs had significantly shorter idea-generation periods (p=0.0004), leading to fewer creative moments (p=0.002). Qualitative analysis of participants' interactions and interviews revealed four different human-LLM collaboration modes supporting various problem-solving strategies. However, a comparative analysis of the generated solutions shows that while LLMs can help generate more correct and functional code, their solutions contain roughly the same number of ideas as participant-generated ones. Based on our findings, we discuss design implications and considerations for effectively using LLMs to support user creativity.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 3 minor

Summary. The paper reports a within-subject study (N=20 programmers) comparing LLM-assisted vs. unassisted conditions on a creative programming task. It claims LLM use produces significantly shorter idea-generation periods (p=0.0004) and fewer creative moments (p=0.002), identifies four human-LLM collaboration modes, and finds that LLM-assisted solutions are more correct/functional but contain roughly the same number of ideas as unassisted ones.

Significance. If the causal attribution holds, the work provides empirical grounding for how LLM interaction alters the temporal structure of creative ideation in programming and offers design implications for tools that preserve creative moments while improving correctness. The identification of collaboration modes and the idea-count comparison are useful for distinguishing process effects from outcome effects.

major comments (2)
  1. [Methods / Experimental Design] The central statistical claims rest on a within-subject comparison, yet the manuscript provides no details on condition counterbalancing, session order randomization, or mixed-effects modeling to test for sequence-by-condition interactions. Without these, the reported shortening of idea-generation periods (p=0.0004) and reduction in creative moments (p=0.002) cannot be unambiguously attributed to LLM presence rather than learning or familiarity effects across sessions.
  2. [Methods / Results] The definitions and operationalization of 'idea-generation periods' and 'creative moments' are not fully specified (e.g., exact coding criteria, temporal granularity, or how they were distinguished from general problem-solving time). This measurement validity issue directly affects the interpretation of the p-values and the claim that shorter periods lead to fewer creative moments.
minor comments (3)
  1. [Results / Qualitative Analysis] The qualitative analysis identifies four collaboration modes but supplies limited illustrative quotes or interaction excerpts, making it difficult to evaluate how the modes map onto the observed quantitative differences.
  2. [Methods] Missing details include inter-rater reliability statistics for the qualitative coding, data exclusion criteria, and the exact statistical tests (e.g., whether paired t-tests or non-parametric equivalents were used given the within-subject structure).
  3. [Results / Solution Comparison] The comparative solution analysis states that LLM solutions contain 'roughly the same number of ideas' but does not report the precise counts, variance, or statistical test used for this equivalence claim.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their detailed and constructive comments on our manuscript. We have carefully considered each point and revised the paper to improve clarity on the experimental design and measurement procedures. Below we respond point-by-point.

read point-by-point responses

  1. Referee: [Methods / Experimental Design] The central statistical claims rest on a within-subject comparison, yet the manuscript provides no details on condition counterbalancing, session order randomization, or mixed-effects modeling to test for sequence-by-condition interactions. Without these, the reported shortening of idea-generation periods (p=0.0004) and reduction in creative moments (p=0.002) cannot be unambiguously attributed to LLM presence rather than learning or familiarity effects across sessions.

    Authors: We agree that the manuscript would benefit from more explicit details on these aspects of the experimental design. In the revised version, we have added a new subsection 'Experimental Design and Procedure' that specifies: (1) conditions were counterbalanced using a Latin square to ensure equal distribution of order across participants, (2) the order of sessions was randomized for each participant, and (3) we employed linear mixed-effects models with session order and condition as fixed effects, including their interaction, to assess potential sequence effects. The interaction term was not significant (p > 0.1), supporting that the observed effects are attributable to the LLM condition rather than order. We have also included the full model specifications and results in the supplementary materials. revision: yes

  2. Referee: [Methods / Results] The definitions and operationalization of 'idea-generation periods' and 'creative moments' are not fully specified (e.g., exact coding criteria, temporal granularity, or how they were distinguished from general problem-solving time). This measurement validity issue directly affects the interpretation of the p-values and the claim that shorter periods lead to fewer creative moments.

    Authors: We acknowledge that the operational definitions could be more precise in the original submission. We have revised the 'Data Analysis' section to provide detailed coding criteria: Idea-generation periods were identified from think-aloud transcripts as sequences of at least 30 seconds involving the articulation of novel programming concepts or approaches, coded at 10-second intervals by two independent raters (Cohen's kappa = 0.82). Creative moments were defined as discrete events where participants expressed or demonstrated a shift to a new idea or solution path, distinguished from routine problem-solving by the presence of 'aha' verbalizations or abrupt changes in code structure. We have added examples from the data and clarified how these were separated from implementation time. These revisions strengthen the validity of the reported statistical findings. revision: yes

Circularity Check

0 steps flagged

No significant circularity in empirical study

full rationale

This paper reports results from a within-subject empirical study (N=20) using standard statistical tests for p-values on idea-generation periods and creative moments, plus thematic analysis of interviews and solution comparisons. No mathematical derivations, equations, fitted parameters presented as predictions, or self-citation chains underpin the central claims. All findings rest on observed participant data and conventional analysis methods rather than any self-referential reduction or renaming of inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claims rest on standard assumptions of statistical testing and qualitative coding rather than new parameters or invented entities.

axioms (2)
  • standard math Standard assumptions of within-subject statistical tests (normality, independence of observations after counterbalancing)
    Invoked implicitly when reporting p=0.0004 and p=0.002 without further qualification.
  • domain assumption Qualitative themes from retrospective interviews accurately reflect participants' creative processes
    Required for the four collaboration modes and the interpretation of fewer creative moments.

pith-pipeline@v0.9.0 · 5490 in / 1266 out tokens · 45966 ms · 2026-05-14T17:36:53.267947+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

84 extracted references · 50 canonical work pages · 2 internal anchors

  1. [1]

    Selcuk Acar, Cyndi Burnett, and John F. Cabra. 2017. Ingredients of Creativity: Originality and More.Creativity Research Journal29 (2017), 133 –

    2017

  2. [2]

    doi:10.1080/10400419.2017.1302776

    work page doi:10.1080/10400419.2017.1302776 2017

  3. [3]

    Teresa M Amabile. 1983. The social psychology of creativity: a componential conceptualization.Journal of personality and social psychology45, 2 (1983), 357

    1983

  4. [4]

    Olson, and Zohreh Sharafi

    Mahta Amini, Jay A. Olson, and Zohreh Sharafi. 2024. Coding with a Creative Twist: Investigating the Link Between Creativity Scores and Problem-solving Strategies.2024 IEEE/ACM 46th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) (2024), 21–25. doi:10.1145/3639476.3639766 Manuscript submitted to ACM Exploring t...

    work page doi:10.1145/3639476.3639766 2024

  5. [6]

    Barrett R Anderson, Jash Hemant Shah, and Max Kreminski. 2024. Homogenization Effects of Large Language Models on Human Creative Ideation. Proceedings of the 16th Conference on Creativity & Cognition(2024). doi:10.1145/3635636.3656204

    work page doi:10.1145/3635636.3656204 2024

  6. [7]

    Tyler Angert, Miroslav Suzara, Jenny Han, Christopher Pondoc, and Hariharan Subramonyam. 2023. Spellburst: A Node-based Interface for Exploratory Creative Coding with Natural Language Prompts. InProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology(San Francisco, CA, USA)(UIST ’23). Association for Computing Machinery, New...

    work page arXiv 2023

  7. [8]

    Ars Technica Staff. 2025. Developer Survey Shows Trust in AI Coding Tools Is Falling as Usage Rises.Ars Technica(July 2025). https://arstechnica. com/ai/2025/07/developer-survey-shows-trust-in-ai-coding-tools-is-falling-as-usage-rises/ [Online; accessed 2025–09–12]

    2025

  8. [9]

    Mia Magdalena Bangerl, Leonie Disch, Tamara David, and Viktoria Pammer-Schindler. 2025. CreAItive Collaboration? Users’ Misjudgment of AI-Creativity Affects Their Collaborative Performance. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Association for Computing Machinery, New York, NY, USA, Article 195, 17 pages...

    work page doi:10.1145/3706598.3713886 2025

  9. [10]

    Bird, Denae Ford, Thomas Zimmermann, Nicole Forsgren, Eirini Kalliamvakou, Travis Lowdermilk, and Idan Gazit

    C. Bird, Denae Ford, Thomas Zimmermann, Nicole Forsgren, Eirini Kalliamvakou, Travis Lowdermilk, and Idan Gazit. 2022. Taking Flight with Copilot.Queue20 (2022), 35 – 57. doi:10.1145/3582083

    work page doi:10.1145/3582083 2022

  10. [11]

    Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology.Qualitative research in psychology3, 2 (2006), 77–101

    2006

  11. [12]

    Tuhin Chakrabarty, Philippe Laban, Divyansh Agarwal, Smaranda Muresan, and Chien-Sheng Wu. 2024. Art or artifice? large language models and the false promise of creativity. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–34

    2024

  12. [13]

    Tuhin Chakrabarty, Vishakh Padmakumar, Faeze Brahman, and S. Muresan. 2023. Creativity Support in the Age of Large Language Models: An Empirical Study Involving Emerging Writers.ArXivabs/2309.12570 (2023). doi:10.48550/arXiv.2309.12570

    work page doi:10.48550/arxiv.2309.12570 2023

  13. [14]

    Tuhin Chakrabarty, Vishakh Padmakumar, Faeze Brahman, and S. Muresan. 2024. Creativity Support in the Age of Large Language Models: An Empirical Study Involving Professional Writers.Proceedings of the 16th Conference on Creativity & Cognition(2024). doi:10.1145/3635636.3656201

    work page doi:10.1145/3635636.3656201 2024

  14. [15]

    Chen and J

    Z. Chen and J. Chan. 2024. Large Language Model in Creative Work: The Role of Collaboration Modality and User Expertise.Management Science 70 (2024), 9101–9117. doi:10.1287/mnsc.2023.03014

    work page doi:10.1287/mnsc.2023.03014 2024

  15. [16]

    Erin Cherry and Celine Latulipe. 2014. Quantifying the creativity support of digital tools through the creativity support index.ACM Transactions on Computer-Human Interaction21, 4 (2014), 1–25

    2014

  16. [17]

    G. E. Corazza, Sergio Agnoli, and Serena Mastria. 2022. The Dynamic Creativity Framework.European Psychologist(2022). doi:10.1027/1016- 9040/a000473

    work page doi:10.1027/1016- 2022

  17. [19]

    Doshi and O

    A. Doshi and O. Hauser. 2024. Generative AI enhances individual creativity but reduces the collective diversity of novel content.Science Advances 10 (2024). doi:10.1126/sciadv.adn5290

    work page doi:10.1126/sciadv.adn5290 2024

  18. [20]

    Wu, Rui Zheng, Ming bo Wen, Rongxiang Weng, Jingang Wang, Xunliang Cai, Tao Gui, Xipeng Qiu, Qi Zhang, and Xuanjing Huang

    Shihan Dou, Haoxiang Jia, Shenxi Wu, Huiyuan Zheng, Weikang Zhou, Muling Wu, Mingxu Chai, Jessica Fan, Caishuang Huang, Yunbo Tao, Yan Liu, Enyu Zhou, Ming Zhang, Yuhao Zhou, Y. Wu, Rui Zheng, Ming bo Wen, Rongxiang Weng, Jingang Wang, Xunliang Cai, Tao Gui, Xipeng Qiu, Qi Zhang, and Xuanjing Huang. 2024. What’s Wrong with Your Code Generated by Large Lan...

    work page doi:10.48550/arxiv.2407.06153 2024

  19. [21]

    Kevin Dunnell, Trudy Painter, Andrew Stoddard, and Andrew Lippman. 2023. Latent Lab: Large Language Models for Knowledge Exploration. ArXivabs/2311.13051 (2023). https://api.semanticscholar.org/CorpusID:259859652

    work page arXiv 2023

  20. [22]

    Rocchesso, Steven W

    Giulia Di Fede, D. Rocchesso, Steven W. Dow, and S. Andolina. 2022. The Idea Machine: LLM-based Expansion, Rewriting, Combination, and Suggestion of Ideas.Proceedings of the 14th Conference on Creativity and Cognition(2022). doi:10.1145/3527927.3535197

    work page doi:10.1145/3527927.3535197 2022

  21. [23]

    Nicholas D Fila, Senay Purzer, and Paul D Mathis. 2014. I’m not the creative type: Barriers to student creativity within engineering innovation projects. In2014 ASEE Annual Conference & Exposition. 24–831

    2014

  22. [24]

    Karan Girotra, Lennart Meincke, Gideon Nave, Christian Terwiesch, and Karl T. Ulrich. 2024. Using Large Language Models for Idea Generation in Innovation.A vailable at SSRN 4526071(September 2024). https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4526071

    2024

  23. [25]

    Ross, Michael J

    Gabriel Enrique Gonzalez, Darío Andrés Silva Moran, Stephanie Houde, Jessica He, Steven I. Ross, Michael J. Muller, Siya Kunde, and Justin D. Weisz

  24. [26]

    InIUI Workshops

    Collaborative Canvas: A Tool for Exploring LLM Use in Group Ideation Tasks. InIUI Workshops. https://api.semanticscholar.org/CorpusID: 269088798

  25. [27]

    The Dynamics of Creativity in Software Development

    D. Graziotin. 2013. The Dynamics of Creativity in Software Development.ArXivabs/1305.6045 (2013). doi:10.6084/m9.figshare.703568

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.6084/m9.figshare.703568 2013

  26. [28]

    Luyten, Joost Vennekens, and Kris Aerts

    Wouter Groeneveld, L. Luyten, Joost Vennekens, and Kris Aerts. 2021. Exploring the Role of Creativity in Software Engineering.2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS)(2021), 1–9. doi:10.1109/icse-seis52602.2021.00009

    work page doi:10.1109/icse-seis52602.2021.00009 2021

  27. [29]

    Luyten, Joost Vennekens, and Kris Aerts

    Wouter Groeneveld, L. Luyten, Joost Vennekens, and Kris Aerts. 2023. Students’ and professionals’ perceived creativity in software engineering: a comparative study.European Journal of Engineering Education48 (2023), 1351 – 1368. doi:10.1080/03043797.2023.2294126

    work page doi:10.1080/03043797.2023.2294126 2023

  28. [30]

    Sarah Harvey and James W Berry. 2023. Toward a meta-theory of creativity forms: How novelty and usefulness shape creativity.Academy of Management Review48, 3 (2023), 504–529

    2023

  29. [31]

    Gadiraju

    Gaole He, Gianluca Demartini, and U. Gadiraju. 2025. Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistant.Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems(2025). doi:10.1145/3706598.3713218 Manuscript submitted to ACM 30 Trovato et al

    work page doi:10.1145/3706598.3713218 2025

  30. [32]

    Heyman, S

    J. Heyman, S. Rick, G. Giacomelli, H. Wen, R. Laubacher, N. Taubenslag, M. Knicker, Y. Jeddi, P. Ragupathy, J. Curhan, and T. Malone. 2024. Supermind Ideator: How scaffolding Human-AI collaboration can increase creativity.Collective Intelligence3 (2024). doi:10.1145/3643562.3672611

    work page doi:10.1145/3643562.3672611 2024

  31. [33]

    Dong Huang, Qingwen Bu, J Zhang, Xiaofei Xie, Junjie Chen, and Heming Cui. 2023. Bias Testing and Mitigation in LLM-based Code Generation. ACM Transactions on Software Engineering and Methodology35 (2023), 1 – 31. doi:10.1145/3724117

    work page doi:10.1145/3724117 2023

  32. [34]

    Sarah Inman, Sarah D’Angelo, and Bogdan Vasilescu. 2024. Developer Productivity for Humans, Part 8: Creativity in Software Engineering.IEEE Softw.41 (2024), 11–16. doi:10.1109/ms.2023.3340831

    work page doi:10.1109/ms.2023.3340831 2024

  33. [35]

    Anna Kantosalo and Sirpa Riihiaho. 2019. Mapping the landscape of creativity support tools in HCI. InProceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–14

    2019

  34. [36]

    Siegel, J’anos Kram’ar, Jonah Brown-Cohen, Samuel Albanie, Jannis Bulian, Rishabh Agarwal, David Lindner, Yunhao Tang, Noah D

    Zachary Kenton, Noah Y. Siegel, J’anos Kram’ar, Jonah Brown-Cohen, Samuel Albanie, Jannis Bulian, Rishabh Agarwal, David Lindner, Yunhao Tang, Noah D. Goodman, and Rohin Shah. 2024. On scalable oversight with weak LLMs judging strong LLMs.ArXivabs/2407.04622 (2024). doi:10.48550/arxiv.2407.04622

    work page doi:10.48550/arxiv.2407.04622 2024

  35. [37]

    Bak, Jing Yao, Jianxun Lian, Muhua Huang, Shitong Duan, and Xing Xie

    Hyunjin Kim, Xiaoyuan Yi, J. Bak, Jing Yao, Jianxun Lian, Muhua Huang, Shitong Duan, and Xing Xie. 2025. The Road to Artificial SuperIntelligence: A Comprehensive Survey of Superalignment.SuperIntelligence - Robotics - Safety & Alignment(2025). doi:10.70777/si.v2i1.13963

    work page doi:10.70777/si.v2i1.13963 2025

  36. [38]

    A. Lasisi. 2016. Creativity in software engineering : a systematic literature review. (2016)

    2016

  37. [39]

    Mina Lee, Percy Liang, and Qian Yang. 2022. CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities. InProceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–19

    2022

  38. [40]

    Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, Kai Shu, Lu Cheng, and Huan Liu. 2025. From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Christ...

    work page doi:10.18653/v1/2025.emnlp-main.138 2025

  39. [41]

    Paul Luo Li, Amy J Ko, and Andrew Begel. 2020. What distinguishes great software engineers?Empirical Software Engineering25, 1 (2020), 322–352

    2020

  40. [42]

    Paul Luo Li, Amy J Ko, and Jiamin Zhu. 2015. What makes a great software engineer?. In2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, Vol. 1. IEEE, 700–710

    2015

  41. [43]

    Lim and S

    G. Lim and S. Perrault. 2024. Rapid AIdeation: Generating Ideas With the Self and in Collaboration With Large Language Models.arXiv preprint arXiv:2403.12928(2024). doi:10.48550/arXiv.2403.12928

    work page doi:10.48550/arxiv.2403.12928 2024

  42. [44]

    Ko, Will Jernigan, Alannah Oleson, Christopher J

    Dastyni Loksa, Amy J. Ko, Will Jernigan, Alannah Oleson, Christopher J. Mendez, and Margaret M. Burnett. 2016. Programming, Problem Solving, and Self-Awareness: Effects of Explicit Guidance. InProceedings of the 2016 CHI Conference on Human Factors in Computing Systems(San Jose, California, USA)(CHI ’16). Association for Computing Machinery, New York, NY,...

    work page doi:10.1145/2858036.2858252 2016

  43. [45]

    Luminate. 2025. Luminate: Entertainment Industry Data, Analytics & Insights. https://luminatedata.com/ Accessed: 2025-09-11

    2025

  44. [46]

    H. R. Maharani, Sukestiyarno Sukestiyarno, and B. Waluya. 2017. Creative Thinking Process Based on Wallas Model in Solving Mathematics Problem. 1 (2017), 177–184. doi:10.12928/IJEME.V1I2.5783

    work page doi:10.12928/ijeme.v1i2.5783 2017

  45. [47]

    Marron and M

    Tali R. Marron and M. Faust. 2019. Measuring spontaneous processes in creativity research.Current Opinion in Behavioral Sciences27 (2019), 64–70. doi:10.1016/j.cobeha.2018.09.009

    work page doi:10.1016/j.cobeha.2018.09.009 2019

  46. [48]

    Mayseless, A

    N. Mayseless, A. Eran, and S. Shamay-Tsoory. 2015. Generating original ideas: The neural underpinning of originality.NeuroImage116 (2015), 232–239. doi:10.1016/j.neuroimage.2015.05.030

    work page doi:10.1016/j.neuroimage.2015.05.030 2015

  47. [49]

    Mohammadi

    B. Mohammadi. 2024. Creativity Has Left the Chat: The Price of Debiasing Language Models.arXiv preprint arXiv:2406.05587(2024). doi:10.48550/ arXiv.2406.05587

    work page arXiv 2024

  48. [50]

    Njoku, Mahta Amini, and Zohreh Sharafi

    A. Njoku, Mahta Amini, and Zohreh Sharafi. 2024. Innovating Coding: Evaluating the Impact of Innovative Thinking in Programming.2024 IEEE/ACM 32nd International Conference on Program Comprehension (ICPC)(2024), 241–245. doi:10.1145/3643916.3644397

    work page doi:10.1145/3643916.3644397 2024

  49. [51]

    Sunday Oladele, Fiyin Iwa, et al. 2025. The Future of Work: Balancing Automation and Human Creativity in the AI Workplace Authors.Fiyin, The Future of Work: Balancing Automation and Human Creativity in the AI Workplace Authors (April 22, 2025)(2025)

    2025

  50. [52]

    Felix Pinkow. 2022. Creative cognition: A multidisciplinary and integrative framework of creative thinking.Creativity and Innovation Management (2022). doi:10.1111/caim.12541

    work page doi:10.1111/caim.12541 2022

  51. [53]

    Ketai Qiu, Niccolò Puccinelli, Matteo Ciniselli, and Luca Di Grazia. 2024. From Today’s Code to Tomorrow’s Symphony: The AI Transformation of Developer’s Routine by 2030.ACM Transactions on Software Engineering and Methodology34 (2024), 1 – 17. doi:10.1145/3709353

    work page doi:10.1145/3709353 2024

  52. [54]

    Marissa Radensky, Simra Shahid, Raymond Fok, Pao Siangliulue, Tom Hope, and Daniel S. Weld. 2024. Scideator: Human-LLM Scientific Idea Generation Grounded in Research-Paper Facet Recombination.ArXivabs/2409.14634 (2024). https://api.semanticscholar.org/CorpusID:272827497

    work page arXiv 2024

  53. [55]

    Leon Reicherts, Zelun Tony Zhang, Elisabeth von Oswald, Yuanting Liu, Yvonne Rogers, and Mariam Hassib. 2025. AI, help me think—but for myself: Assisting people in complex decision-making by providing different kinds of cognitive support. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–19

    2025

  54. [56]

    Shuo Ren, Daya Guo, Shuai Lu, Long Zhou, Shujie Liu, Duyu Tang, Neel Sundaresan, Ming Zhou, Ambrosio Blanco, and Shuai Ma. 2020. Codebleu: a method for automatic evaluation of code synthesis.arXiv preprint arXiv:2009.10297(2020)

    work page internal anchor Pith review arXiv 2020

  55. [57]

    Rietschel, F

    M. Rietschel, F. Guo, and K. Steinfeld. 2024. Mediating Modes of Thought: LLM’s for design scripting.arXiv preprint arXiv:2411.14485(2024). doi:10.48550/arXiv.2411.14485

    work page doi:10.48550/arxiv.2411.14485 2024

  56. [58]

    Runco and R

    M. Runco and R. E. Charles. 1993. Judgments of originality and appropriateness as predictors of creativity.Personality and Individual Differences15 (1993), 537–546. doi:10.1016/0191-8869(93)90337-3 Manuscript submitted to ACM Exploring the Impact of LLM Interaction on the Creative Process of Programming 31

    work page doi:10.1016/0191-8869(93)90337-3 1993

  57. [59]

    Bowden, and Mark Beeman

    Carola Salvi, Emanuela Bricolo, John Kounios, Edward M. Bowden, and Mark Beeman. 2016. Insight solutions are correct more often than analytic solutions.Thinking & Reasoning22 (2016), 443 – 460. https://api.semanticscholar.org/CorpusID:9947798

    2016

  58. [60]

    Bhattacharya

    Simone Sandkühler and J. Bhattacharya. 2008. Deconstructing Insight: EEG Correlates of Insightful Problem Solving.PLoS ONE3 (2008). doi:10.1371/journal.pone.0001459

    work page doi:10.1371/journal.pone.0001459 2008

  59. [61]

    Setiawani, A

    S. Setiawani, A. Fatahillah, Dafik, E. Oktavianingtyas, and D. Wardani. 2019. The students’ creative thinking process in solving mathematics problem based on wallas’ stages.IOP Conference Series: Earth and Environmental Science243 (2019). doi:10.1088/1755-1315/243/1/012052

    work page doi:10.1088/1755-1315/243/1/012052 2019

  60. [62]

    Shaer, A

    O. Shaer, A. Cooper, O. Mokryn, A. Kun, and H. Shoshan. 2024. AI-Augmented Brainwriting: Investigating the use of LLMs in group ideation. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. doi:10.1145/3613904.3642414

    work page doi:10.1145/3613904.3642414 2024

  61. [63]

    Ben Shneiderman. 2002. Creativity support tools.Commun. ACM45, 10 (2002), 116–120

    2002

  62. [64]

    Ben Shneiderman. 2007. Creativity support tools: accelerating discovery and innovation.Commun. ACM50, 12 (2007), 20–32

    2007

  63. [65]

    Sternberg and Sareh Karami

    R. Sternberg and Sareh Karami. 2021. An 8P Theoretical Framework for Understanding Creativity and Theories of Creativity.The Journal of Creative Behavior(2021). doi:10.1002/jocb.516

    work page doi:10.1002/jocb.516 2021

  64. [66]

    Aben, Axel Cleeremans, and E

    Hans Stuyck, B. Aben, Axel Cleeremans, and E. Bussche. 2021. The Aha! moment: Is insight a different form of problem solving?Consciousness and Cognition90 (2021). doi:10.1016/j.concog.2020.103055

    work page doi:10.1016/j.concog.2020.103055 2021

  65. [67]

    S. Suh, M. Chen, B. Min, T. Li, and H. Xia. 2023. Luminate: Structured Generation and Exploration of Design Space with Large Language Models for Human-AI Co-Creation. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems

    2023

  66. [68]

    Sangho Suh, Meng Chen, Bryan Min, Toby Jia-Jun Li, and Haijun Xia. 2023. Luminate: Structured Generation and Exploration of Design Space with Large Language Models for Human-AI Co-Creation.Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems(2023). https://api.semanticscholar.org/CorpusID:269752103

    2023

  67. [69]

    Sascha Topolinski and Rolf Reber. 2010. Gaining insight into the “Aha” experience.Current Directions in Psychological Science19, 6 (2010), 402–405

    2010

  68. [70]

    1926.The Art of Thought

    Graham Wallas. 1926.The Art of Thought. Harcourt Brace

    1926

  69. [71]

    S. G. Wallas. 1970. The Art of Thought. Harcourt Brace & World Inc. 1926. https://api.semanticscholar.org/CorpusID:204720596

    1970

  70. [72]

    It Felt Like Having a Second Mind

    Qian Wan, Si-Yuan Hu, Yu Zhang, Pi-Hui Wang, Bo Wen, and Zhicong Lu. 2023. "It Felt Like Having a Second Mind": Investigating Human- AI Co-creativity in Prewriting with Large Language Models.Proceedings of the ACM on Human-Computer Interaction8 (2023), 1 – 26. https: //api.semanticscholar.org/CorpusID:259991355

    2023

  71. [73]

    A. Wang, Z. Yin, Y. Hu, Y. Mao, and P. Hui. 2024. Exploring the Potential of Large Language Models in Artistic Creation: Collaboration and Reflection on Creative Programming.arXiv preprint arXiv:2402.09750(2024). doi:10.48550/arXiv.2402.09750

    work page doi:10.48550/arxiv.2402.09750 2024

  72. [74]

    Wang, Chinmay Kulkarni, Lauren Wilcox, Michael Terry, and Michael Madaio

    Zijie J. Wang, Chinmay Kulkarni, Lauren Wilcox, Michael Terry, and Michael Madaio. 2024. Farsight: Fostering Responsible AI Awareness During AI Application Prototyping. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems(Honolulu, HI, USA)(CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 976, 40 pages. d...

    work page doi:10.1145/3613904.3642335 2024

  73. [75]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. InProceedings of the 36th International Conference on Neural Information Processing Systems (New Orleans, LA, USA)(NIPS ’22). Curran Associates Inc., Red Hook, NY...

    2022

  74. [76]

    Geraint A. Wiggins. 2006. A preliminary framework for description, analysis and comparison of creative systems.Knowl. Based Syst.19 (2006), 449–458. doi:10.1016/j.knosys.2006.04.009

    work page doi:10.1016/j.knosys.2006.04.009 2006

  75. [77]

    Geraint A. Wiggins. 2019. A Framework for Description, Analysis and Comparison of Creative Systems. (2019), 21–47. doi:10.1007/978-3-319- 43610-4_2

    work page doi:10.1007/978-3-319- 2019

  76. [78]

    Wong, Shangxin Guo, Ching Nam Hang, Siu-Wai Ho, and C

    M. Wong, Shangxin Guo, Ching Nam Hang, Siu-Wai Ho, and C. Tan. 2023. Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review.Entropy25 (2023). doi:10.3390/e25060888

    work page doi:10.3390/e25060888 2023

  77. [79]

    Richard W Woodman, John E Sawyer, and Ricky W Griffin. 1993. Toward a theory of organizational creativity.Academy of management review18, 2 (1993), 293–321

    1993

  78. [80]

    Sutiarso

    Avissa Purnama Yanti, Budi Koestoro, and S. Sutiarso. 2018. The Students’ Creative Thinking Process based on Wallas Theory in Solving Mathematical Problems viewed from Adversity Quotient /Type Climbers.Al-Jabar : Jurnal Pendidikan Matematika(2018). doi:10.24042/AJPM.V9I1.2331

    work page doi:10.24042/ajpm.v9i1.2331 2018

  79. [81]

    Ann Yuan, Andy Coenen, Emily Reif, and Daphne Ippolito. 2022. Wordcraft: Story writing with large language models. InProceedings of the 27th International Conference on Intelligent User Interfaces. 841–852

    2022

  80. [82]

    JD Zamfirescu-Pereira, Eunice Jun, Michael Terry, Qian Yang, and Björn Hartmann. 2025. Beyond code generation: Llm-supported exploration of the program design space. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–17

    2025

Showing first 80 references.