Who's the Leader? Analyzing Novice Workflows in LLM-Assisted Debugging of Machine Learning Code

Ashton Anderson; Emma Zhuang; Jessica Y. Bo; Majeed Kazemitabaar

REVIEW 3 major objections 5 minor 1 cited by

Who's the Leader? Analyzing Novice Workflows in LLM-Assisted Debugging of Machine Learning Code

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read In LLM-assisted ML debugging, who leads the conversation predicts who succeeds.

desk verdict A useful workshop-level study of novice ML debugging with ChatGPT that adds three concrete failure patterns, but the main leading/led-by coding lacks reliability checks, so the headline correlation should be treated as suggestive. read the letter →

arxiv 2505.08063 v1 pith:EIKD34VW submitted 2025-05-12 cs.HC

Jessica Y. Bo , Majeed Kazemitabaar , Emma Zhuang , Ashton Anderson This is my paper · ORCID

classification cs.HC

keywords novice-LLMinteractionsLLM-assistedcodingdebuggingmachinelearningmentalmodelscognitiveengagementover-relianceappropriatereliance

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that, when novices debug machine-learning code with an LLM assistant, the style of interaction matters more than the amount of help used. In a formative study with eight novice ML engineers given a deliberately broken Random Forest script and free access to ChatGPT, the authors categorize every conversation turn as either 'leading' the LLM (asking specific, planned questions) or being 'led by' the LLM (asking open-ended questions and following suggestions). They report that participants who led the LLM generally achieved higher holdout $F1$ scores, while those who delegated broadly tended to over-rely on ChatGPT; they also observe a strong correlation between a short ML knowledge quiz and task performance. The stakes are educational: if a novice's prompting style predicts both performance and the risk of reinforcing wrong mental models, then tools and tutorials can be designed to keep novices in the leading role.

What carries the argument

The load-bearing object is the turn-level coding scheme that divides every user-ChatGPT exchange into 'leading' (specific, planned, hypothesis-driven questions) versus 'led-by' (open-ended questions followed by uncritical acceptance), together with the four-category reliance outcome scheme (rely on self, rely on LLM, over-rely, under-rely) adapted from prior work on appropriate reliance. This scheme is what connects observable prompt language to task performance: the proportion of leading turns is the predictor, and the outcome categories explain why failures happen, whether from blindly following wrong advice or ignoring correct advice. It also exposes three novice-specific failure modes: injecting a wrong 'lead' into a query, being overwhelmed by long lists of suggestions, and struggling to verify which metrics matter.

What would settle it

Re-code the eight transcripts with two or more independent raters who are blind to holdout scores, using the paper's definitions of leading and led-by; if inter-rater agreement is low (for example, Cohen's kappa below 0.6), then the reported link between leading behavior and performance may reflect the coders' impressions rather than a stable behavioral difference. A larger pre-registered replication could also test whether the proportion of leading turns predicts holdout $F1$ score, which would settle the claim beyond the current small sample.

Watch

Extended reading notes

Core claim

The paper's central claim is that novice-LLM interaction in a high-complexity, low-verifiability task falls into two recognizable patterns, and the pattern predicts the quality of reliance. A 'leading' turn is a specific, planned query aimed at testing the user's own hypothesis, such as asking which hyperparameters matter most for a Random Forest; a 'led-by' turn is an open-ended request such as 'What is wrong with this code?' followed by acceptance of whatever ChatGPT suggests. Coding the eight sessions turn by turn, the authors find that four of the five top scorers on the holdout set led ChatGPT in at least half of their turns, while participants who shifted the workload to ChatGPT performed worse. They also map each turn onto reliance outcomes—rely on self, rely on LLM, over-rely, under-rely—and report that over-reliance clusters in led-by sessions while under-reliance appears when novices drive with a faulty hypothesis, such as asking for feature standardization when it is irrelevant to the model. Finally, they report a correlation of $r = .93$ between the pre-task ML quiz score and holdout $F1$ performance, which they interpret as evidence that domain knowledge, not self-reported experience, governs whether LLM assistance is used well.

Load-bearing premise

The paper's central pattern rests on the authors' binary coding of every conversation turn as leading or led-by, and the paper reports no check that independent raters would apply that coding the same way.

Editorial extensions

If this is right

In LLM-assisted debugging, the share of leading turns can be measured and used to identify novices who are likely to over-rely before they finish the task.
Interventions that push novices to form and state a hypothesis before querying should improve both debugging outcomes and the accuracy of reliance, since leading behavior is associated with better holdout performance.
Domain knowledge, as measured by a short quiz, predicts performance with an LLM assistant better than self-reported ML experience, so assessment tools should measure actual knowledge rather than confidence.
LLM interfaces for novices need to handle two distinct failure modes: they should push back on incorrect user 'leads' to prevent over-reliance, and they should help filter broad responses to prevent under-reliance from cognitive overload.
Because participants said they would prompt differently in hindsight, brief prompting tutorials may shift led-by novices toward leading behavior.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

I infer that the leading/led-by distinction is not specific to ML debugging: any task where answers are hard to verify, such as data analysis or scientific writing, should show the same pattern, and the coding scheme could be tested on transcripts from those domains.
The strong quiz-score correlation, despite the small sample, suggests a cheap practical proxy: a short knowledge test could decide when an LLM assistant should offer structured guidance rather than direct answers.
A testable design extension is to make the LLM explicitly label when it disagrees with the user's stated hypothesis; comparing over-reliance rates and post-task quizzes against a vanilla assistant would show whether correcting faulty mental models reduces the harms observed here.
The paper's framing implies that the 'learning' novices reported was mostly low-level syntax; I infer that measuring mental-model change through a post-task debugging quiz would reveal whether leading behavior also causes learning, not just performance.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague, quick take on arXiv:2505.08063. It's a 5-page workshop paper with a smart, small study: eight ML novices debug a deliberately broken Random Forest script with ChatGPT. The new bit is applying the known leading/led-by distinction (Barke, Prather) to ML debugging, plus three named meta-cognitive error patterns — injecting wrong leads into queries, being overwhelmed by broad lists, and failing at verification. Those patterns are the most useful part of the paper; they give designers concrete levers.

The study design is fine for a formative effort: three planted bugs, held-out F1 scoring, a knowledge quiz, think-aloud, interviews. The authors are appropriately cautious about the r=.93 between quiz score and F1 on n=8, and they note the small sample.

The soft spot is the main independent variable. Reliance actions (leading vs led-by) and reliance outcomes (over/under-rely) were coded from notes and transcripts, but there is no codebook, no second coder, no inter-rater reliability, and no blinding to performance. The same people who scored the code also coded the conversations. That means the headline result — four of five top scorers led the LLM in half or more turns — could partly reflect coder expectations. This is a missing validation step, not evidence of bias. The paper does not acknowledge it. For a workshop paper, that is a common and tolerable gap if the claims stay tentative; here they mostly do, though the abstract and some wording ('participants who ... performed worse') reads a bit more causal than the evidence supports.

Bottom line: this is a legitimate formative study that adds useful texture to the novice-LLM interaction literature. It deserves a workshop referee, and I'd be comfortable with it appearing in a workshop venue. If the authors push toward a full archival paper, they need at least a second coder, a pre-registered coding scheme, and ideally a bigger sample. For the reading group, it's a good case study in how small-n qualitative results can be shapeable by coding choices.

Referee Report

3 major / 5 minor

Summary. This paper reports a formative study with eight novice ML engineers who debugged a deliberately bugged Random Forest training script on the UCI Adult dataset with free access to ChatGPT. It develops a binary distinction between 'leading' the LLM (specific, planned queries) and being 'led-by' it (open-ended queries followed blindly), codes every conversation turn into these categories and into reliance outcomes (over-, under-, and correct reliance), and relates these to holdout F1 performance and subjective perceptions. The main empirical claim is that participants who led the LLM tended to achieve better task performance—four of the five top scorers led in at least half of the turns—while led-by participants tended to over-rely and perform worse. The paper also identifies meta-cognitive errors (leading wrong hypotheses, poor filtering of broad output, verification difficulties) and proposes augmentations to the novice-LLM interaction paradigm.

Significance. If the leading/led-by taxonomy is reproducible, the paper offers a useful behavioral marker of cognitive engagement in LLM-assisted debugging and connects observable prompting style to objective task outcomes in a low-verifiability domain. The study is appropriately framed as formative, the authors situate the taxonomy in prior paradigms (Barke et al.'s acceleration/exploration, Prather et al.'s shepherding/drifting) and an established reliance framework, and they use an objective holdout F1 score rather than only self-report. The paper also honestly acknowledges the n=8 limitation and does not overstate the quiz correlation. Its value lies mainly in hypothesis generation for future studies of novice-LLM interaction in ML debugging.

major comments (3)

[Section 3.2] The central independent variable of the study is the binary Reliance Action (leading vs. led-by), yet the coding procedure is described only as 'coded for each conversation turn ... informed by notes and audio transcripts,' with no codebook, annotation guidelines, or inter-rater reliability metric reported. Because the same research team that assessed holdout F1 also performed this coding, the headline result—that four of the five top scorers led the LLM in at least half of the turns—could reflect coder expectations rather than a stable behavioral difference. The same concern applies to Reliance Outcomes, which require judgments about whether ChatGPT's advice was correct. Please provide the coding manual, the number of coders and whether they were blind to task performance, and agreement statistics (e.g., Cohen's kappa) for both coding schemes, or explicitly reframe the paper as reporting formative, coder-dependent observations.
[Section 3.2 (also Figure 1)] The claim that 'participants who asked for general guidance (led-by) and shifted the workload to ChatGPT generally performed worse' is stated as a quasi-causal relationship, but the study is observational and the association between leading proportion and F1 is not accompanied by a statistical test or even a per-participant scatterplot. Given that the same section reports a very strong quiz–F1 correlation (r = .93), initial ML knowledge is a plausible confound driving both prompting style and performance. Please report the per-participant values (leading proportion, quiz score, F1 score), use a non-parametric correlation with a clear caveat, and soften the causal phrasing throughout the Discussion.
[Section 3.2] The statement 'Four of the five top scorers ... were leading ChatGPT in half or more of the conversation turns' is presented without a definition of 'top scorers' or a table of per-participant F1 scores, so a reader cannot assess whether this grouping is robust to reasonable alternatives (e.g., using whether each of the three planted errors was fixed rather than F1, or excluding participants whose modifications made performance worse than the unmodified baseline). Please provide the underlying numbers and justify the grouping.

minor comments (5)

[Figure 1 caption and Section 3.2] The caption says the top bar is reliance outcomes and the bottom bar is actions, while the text says the top bar is reliance actions and the bottom bar is outcomes; please correct the inconsistency.
[Section 3.1] With n = 8, a Pearson r of .93 should be accompanied by a scatter plot and ideally a Spearman correlation, since a single participant could strongly influence the reported value; the current text appropriately de-emphasizes the result, but the figure would make the caveat concrete.
[ACM Reference Format] The reference format line lists 2018, but the paper is dated 2025; please update the year and venue metadata.
[Section 3.2] The label 'Leading Query' for one of the meta-cognitive errors is easily confused with the leading/led-by action categories; consider renaming it (e.g., 'anchoring on a wrong hypothesis') to avoid ambiguity.
[Section 2.2] The five knowledge quiz questions are not included; adding them or providing an OSF/appendix link would improve reproducibility of the quiz measure.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: leading/led-by and reliance outcomes are coded from observed transcripts, and performance is independently measured; the only self-citation is a non-load-bearing conceptual framework.

full rationale

The paper makes no derivation whose conclusion is identical to its premises. Reliance Actions (leading vs. led-by) are a qualitative coding of conversation turns, informed by notes and audio transcripts, and Reliance Outcomes are coded from participants' responses to ChatGPT. The headline correlation with holdout F1 is empirical: the behavior codes and the held-out performance were measured separately, so 'participants who were led-by performed worse' is not true by definition. The knowledge-quiz correlation (r = .93) is computed from pre-task quiz scores and holdout F1, not derived from a fitted parameter. The only self-citation is [7], the authors' prior work, used alongside [23] to name the four Appropriate Reliance outcome categories; this is a conceptual lens rather than an unverified theorem that forces the paper's result, and the categories are also grounded in the independent Schemmer et al. framework. The paper explicitly connects leading/led-by to prior external paradigms (Barke et al.; Prather et al.), so it does not present a prior category as a new derivation. A legitimate methodological weakness—no reported inter-rater reliability or blinding for the turn-level coding—is a validity and reproducibility concern, not circularity, because the coding is an interpretation of data rather than a parameter fitted to the outcome. No equations reduce to inputs, no prediction is a refit, and no uniqueness claim is imported from the authors' prior work. Therefore the study is self-contained as an observational analysis and receives a circularity score of 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters are fitted; the paper is an observational study. The axioms are standard domain assumptions about measurement tools, task validity, and the representativeness of the LLM. No new physical or conceptual entities are postulated; the leading/led-by categories are analytical constructs derived from prior work, not invented entities in the sense of the ledger.

assumptions (4)

domain assumption The five-question ML knowledge quiz accurately estimates participants' actual ML ability despite possible self-report bias.
Invoked in Section 2.2 as a measure to 'estimate their actual abilities for the task', and then used in the Section 3.1 correlation with F1 score.
domain assumption Think-aloud verbalizations and chat logs accurately reflect participants' cognitive processes and decision-making.
The reliance actions are coded 'informed by notes and audio transcripts taken during the study session' (Section 3.2), assuming the think-aloud protocol captures genuine intentions.
domain assumption ChatGPT 4o-mini is representative of typical LLM assistance available to novices.
Stated in footnote 1: 'we use the mini version as it was available on free accounts'; the study's implications for 'LLM assistants' generalize beyond this specific model.
domain assumption The three injected bugs (E1, E2, E3) and the Adult Income dataset provide a task with sufficient complexity and low verifiability to trigger novice-LLM interaction patterns.
The task design in Section 2.1 assumes this debugging task is representative of real ML debugging challenges.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Who's the Leader? Analyzing Novice Workflows in LLM-Assisted Debugging of Machine Learning Code." pith.science (2026). https://pith.science/paper/EIKD34VW

@misc{pith2026250508063,
  author       = {Pith},
  title        = {Pith review of: Who's the Leader? Analyzing Novice Workflows in LLM-Assisted Debugging of Machine Learning Code},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EIKD34VW}},
  note         = {Machine review of arXiv:2505.08063}
}

read the original abstract

While LLMs are often touted as tools for democratizing specialized knowledge to beginners, their actual effectiveness for improving task performance and learning is still an open question. It is known that novices engage with LLMs differently from experts, with prior studies reporting meta-cognitive pitfalls that affect novices' ability to verify outputs and prompt effectively. We focus on a task domain, machine learning (ML), which embodies both high complexity and low verifiability to understand the impact of LLM assistance on novices. Provided a buggy ML script and open access to ChatGPT, we conduct a formative study with eight novice ML engineers to understand their reliance on, interactions with, and perceptions of the LLM. We find that user actions can be roughly categorized into leading the LLM and led-by the LLM, and further investigate how they affect reliance outcomes like over- and under-reliance. These results have implications on novices' cognitive engagement in LLM-assisted tasks and potential negative effects on downstream learning. Lastly, we pose potential augmentations to the novice-LLM interaction paradigm to promote cognitive engagement.

Figures

Figures reproduced from arXiv: 2505.08063 by the authors.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Understanding, Protecting, and Augmenting Human Cognition with Generative AI: A Synthesis of the CHI 2025 Tools for Thought Workshop
cs.HC 2025-08 conditional novelty 4.0 of 10

A synthesis of the CHI 2025 workshop maps research and design opportunities for understanding, protecting, and augmenting human cognition with generative AI.

Reference graph

Works this paper leans on

38 extracted references · 16 canonical work pages · cited by 1 Pith paper

[1]

Saleema Amershi, Andrew Begel, Christian Bird, Robert DeLine, Harald Gall, Ece Kamar, Nachiappan Nagappan, Besmira Nushi, and Thomas Zimmermann. 2019. Software engineering for machine learning: A case study. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). IEEE, 291–300

work page 2019
[2]

Saleema Amershi, Max Chickering, Steven M Drucker, Bongshin Lee, Patrice Simard, and Jina Suh. 2015. Modeltracker: Redesigning performance analysis tools for machine learning. In Proceedings of the 33rd annual ACM conference on human factors in computing systems . 337–346

work page 2015
[3]

Emily Judith Arteaga Garcia, João Felipe Nicolaci Pimentel, Zixuan Feng, Marco Gerosa, Igor Steinmacher, and Anita Sarma. 2024. How to support ml end-user programmers through a conversational agent. InProceedings of the 46th IEEE/ACM International Conference on Software Engineering . 1–12

work page 2024
[4]

Rob Ashmore, Radu Calinescu, and Colin Paterson. 2021. Assuring the machine learning lifecycle: Desiderata, methods, and challenges. ACM Computing Surveys (CSUR) 54, 5 (2021), 1–39

work page 2021
[5]

Shraddha Barke, Michael B James, and Nadia Polikarpova. 2023. Grounded copilot: How programmers interact with code-generating models. Proceedings of the ACM on Programming Languages 7, OOPSLA1 (2023), 85–111

2023
[6]

Barry Becker and Ronny Kohavi. 1996. Adult. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5XW20

doi:10.24432/c5xw20 1996
[7]

Jessica Y Bo, Sophia Wan, and Ashton Anderson. 2024. To Rely or Not to Rely? Evaluating Interventions for Appropriate Reliance on Large Language Models. arXiv preprint arXiv:2412.15584 (2024)

arXiv 2024
[8]

Carrie J Cai and Philip J Guo. 2019. Software developers learning machine learning: Motivations, hurdles, and desires. In 2019 IEEE symposium on visual languages and human-centric computing (VL/HCC) . IEEE, 25–34

work page 2019

Show all 38 references

[9]

Jialun Cao, Meiziniu Li, Ming Wen, and Shing-chi Cheung. 2023. A study on prompt design, advantages and limitations of chatgpt for deep learning program repair. arXiv preprint arXiv:2304.08191 (2023)

2023 arXiv
[10]

Kartik Chandra, Katherine M Collins, Will Crichton, Tony Chen, Tzu-Mao Li, Adrian Weller, Rachit Nigam, Joshua Tenenbaum, and Jonathan Ragan-Kelley
[11]

John Chen, Xi Lu, Yuzhou Du, Michael Rejtig, Ruth Bagley, Mike Horn, and Uri Wilensky. 2024. Learning agent-based modeling with LLM companions: Experiences of novices and experts using ChatGPT & NetLogo chat. InProceedings of the CHI Conference on Human Factors in Computing Sy...

2024
[12]

J Andrés Díaz-Pace, Antonela Tommasel, and Rafael Capilla. 2024. Helping Novice Architects to Make Quality Design Decisions Using an LLM-Based Assistant. In European Conference on Software Architecture. Springer, 324–332

2024
[13]

Kraig Finstad. 2010. The usability metric for user experience. Interacting with computers 22, 5 (2010), 323–327

2010
[14]

Ahmad Ghazal, Tilmann Rabl, Minqing Hu, Francois Raab, Meikel Poess, Alain Crolotte, and Hans-Arno Jacobsen. 2013. Bigbench: Towards an industry stan- dard benchmark for big data analytics. In Proceedings of the 2013 ACM SIGMOD international conference on Management of data . ...

2013
[15]

Majeed Kazemitabaar, Xinying Hou, Austin Henley, Barbara Jane Ericson, David Weintrop, and Tovi Grossman. 2023. How novices use LLM-based code generators to solve CS1 coding tasks in a self-paced learning environment. In Proceedings of the 23rd Koli Calling International Confe...

2023
[16]

Sanjay Krishnan and Eugene Wu. 2017. Palm: Machine learning explanations for iterative debugging. In Proceedings of the 2Nd workshop on human-in-the-loop data analytics. 1–6

2017
[17]

Francesca Lucchetti, Zixuan Wu, Arjun Guha, Molly Q Feldman, and Carolyn Jane Anderson. 2024. Substance Beats Style: Why Beginning Students Fail to Code with LLMs. arXiv preprint arXiv:2410.19792 (2024)

2024 arXiv
[18]

Qianou Ma, Weirui Peng, Hua Shen, Kenneth Koedinger, and Tongshuang Wu
[19]

Nadia Nahar, Haoran Zhang, Grace Lewis, Shurui Zhou, and Christian Kästner
[20]

arXiv preprint arXiv:2409.08775 (2024)

What you say= what you want? Teaching humans to articulate requirements for LLMs. arXiv preprint arXiv:2409.08775 (2024)

2024 arXiv
[21]

It’s Weird That it Knows What I Want

James Prather, Brent N Reeves, Paul Denny, Brett A Becker, Juho Leinonen, Andrew Luxton-Reilly, Garrett Powell, James Finnie-Ansley, and Eddie Antonio Santos. 2023. “It’s Weird That it Knows What I Want”: Usability and Interactions with Copilot for Novice Programmers. ACM Tran...

2023
[22]

James Prather, Brent N Reeves, Juho Leinonen, Stephen MacNeil, Arisoa S Randri- anasolo, Brett A Becker, Bailey Kimmel, Jared Wright, and Ben Briggs. 2024. The widening gap: The benefits and harms of generative ai for novice programmers. In Proceedings of the 2024 ACM Conferen...

2024
[23]

Sydney Nguyen, Hannah McLean Babe, Yangtian Zi, Arjun Guha, Carolyn Jane Anderson, and Molly Q Feldman. 2024. How Beginning Programmers and Code LLMs (Mis) read Each Other. In Proceedings of the CHI Conference on Human Factors in Computing Systems . 1–26

2024
[24]

Eldon Schoop, Forrest Huang, and Bjoern Hartmann. 2021. Umlaut: Debug- ging deep learning programs using program structure and model behavior. In Proceedings of the 2021 CHI conference on human factors in computing systems . 1–16

2021
[25]

Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R Johnston, et al. 2023. Towards understanding sycophancy in language models. arXiv preprint arXiv:2310.13548 (2023)

2023 arXiv
[26]

Max Schemmer, Niklas Kuehl, Carina Benz, Andrea Bartos, and Gerhard Satzger
[27]

In Proceedings of the 28th International Conference on Intelligent User Interfaces

Appropriate reliance on AI advice: Conceptualization and the effect of explanations. In Proceedings of the 28th International Conference on Intelligent User Interfaces. 410–422
[28]

Qiaosi Wang, Koustuv Saha, Eric Gregori, David Joyner, and Ashok Goel. 2021. Towards mutual theory of mind in human-ai interaction: How language reflects what students perceive about a virtual teaching assistant. In Proceedings of the 2021 CHI conference on human factors in co...

2021
[29]

Rose Wang, Qingyang Zhang, Carly Robinson, Susanna Loeb, and Dorottya Demszky. 2024. Bridging the novice-expert gap via models of decision-making: A case study on remediating math mistakes. In Proceedings of the 2024 Conference of the North American Chapter of the Association ...

2024
[30]

Lev Tankelevitch, Viktor Kewenig, Auste Simkute, Ava Elizabeth Scott, Advait Sarkar, Abigail Sellen, and Sean Rintel. 2024. The metacognitive demands and opportunities of generative AI. In Proceedings of the CHI Conference on Human Factors in Computing Systems . 1–24

2024
[31]

Jiessie Tie, Bingsheng Yao, Tianshi Li, Syed Ishtiaque Ahmed, Dakuo Wang, and Shurui Zhou. 2024. LLMs are Imperfect, Then What? An Empirical Study on LLM Failures in Software Engineering. arXiv preprint arXiv:2411.09916 (2024)

2024 arXiv
[32]

Shao Zhang, Xihuai Wang, Wenhao Zhang, Yongshan Chen, Landi Gao, Dakuo Wang, Weinan Zhang, Xinbing Wang, and Ying Wen. 2024. Mutual theory of mind in human-ai collaboration: An empirical study with llm-driven ai agents in a real-time shared workspace task. arXiv preprint arXiv...

2024 arXiv
[33]

Zibin Zheng, Kaiwen Ning, Yanlin Wang, Jingwen Zhang, Dewu Zheng, Mingxi Ye, and Jiachi Chen. 2023. A survey of large language models for code: Evolution, benchmarking, and future trends. arXiv preprint arXiv:2311.10372 (2023)

2023 arXiv
[34]

Qian Yang, Jina Suh, Nan-Chen Chen, and Gonzalo Ramos. 2018. Grounding in- teractive machine learning tool design in how non-experts actually build models. In Proceedings of the 2018 designing interactive systems conference . 573–584

2018
[35]

JD Zamfirescu-Pereira, Richmond Y Wong, Bjoern Hartmann, and Qian Yang
[36]

In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Why Johnny can’t prompt: how non-AI experts try (and fail) to design LLM prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21

2023
[2023]

In 2023 IEEE/ACM 2nd Interna- tional Conference on AI Engineering–Software Engineering for AI (CAIN)

A meta-summary of challenges in building products with ml components– collecting experiences from 4758+ practitioners. In 2023 IEEE/ACM 2nd Interna- tional Conference on AI Engineering–Software Engineering for AI (CAIN) . IEEE, 171–183

2023
[2024]

arXiv preprint arXiv:2403.05334 (2024)

Watchat: Explaining perplexing programs by debugging mental models. arXiv preprint arXiv:2403.05334 (2024)

2024 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Saleema Amershi, Andrew Begel, Christian Bird, Robert DeLine, Harald Gall, Ece Kamar, Nachiappan Nagappan, Besmira Nushi, and Thomas Zimmermann. 2019. Software engineering for machine learning: A case study. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). IEEE, 291–300

work page 2019

[2] [2]

Saleema Amershi, Max Chickering, Steven M Drucker, Bongshin Lee, Patrice Simard, and Jina Suh. 2015. Modeltracker: Redesigning performance analysis tools for machine learning. In Proceedings of the 33rd annual ACM conference on human factors in computing systems . 337–346

work page 2015

[3] [3]

Emily Judith Arteaga Garcia, João Felipe Nicolaci Pimentel, Zixuan Feng, Marco Gerosa, Igor Steinmacher, and Anita Sarma. 2024. How to support ml end-user programmers through a conversational agent. InProceedings of the 46th IEEE/ACM International Conference on Software Engineering . 1–12

work page 2024

[4] [4]

Rob Ashmore, Radu Calinescu, and Colin Paterson. 2021. Assuring the machine learning lifecycle: Desiderata, methods, and challenges. ACM Computing Surveys (CSUR) 54, 5 (2021), 1–39

work page 2021

[5] [5]

Shraddha Barke, Michael B James, and Nadia Polikarpova. 2023. Grounded copilot: How programmers interact with code-generating models. Proceedings of the ACM on Programming Languages 7, OOPSLA1 (2023), 85–111

2023

[6] [6]

Barry Becker and Ronny Kohavi. 1996. Adult. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5XW20

doi:10.24432/c5xw20 1996

[7] [7]

Jessica Y Bo, Sophia Wan, and Ashton Anderson. 2024. To Rely or Not to Rely? Evaluating Interventions for Appropriate Reliance on Large Language Models. arXiv preprint arXiv:2412.15584 (2024)

arXiv 2024

[8] [8]

Carrie J Cai and Philip J Guo. 2019. Software developers learning machine learning: Motivations, hurdles, and desires. In 2019 IEEE symposium on visual languages and human-centric computing (VL/HCC) . IEEE, 25–34

work page 2019