{"id":"4c94db5d-5eeb-44f8-9efe-99fefc114a82","arxiv_id":"2505.08063","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"In an eight-person formative study, novice ML engineers who actively led the ChatGPT debugging conversation outperformed those who followed it, with patterns of over- and under-reliance.","lead":"A small study of eight machine learning novices debugging a broken ML script with ChatGPT found that participants who steered the conversation performed better than those who followed the chat tool. The results show how LLM help can lead beginners into over-reliance instead of building understanding.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's central claim rests on a binary leading/led-by coding that has no reported inter-rater reliability or blind rating; because the same team scored performance and behavior, the observed correlation may reflect coder expectations rather than a stable behavioral difference.","rationale":"Good-faith reading: the paper is a clearly scoped formative study and does not overstate its statistical power; it explicitly says it does not assign strong implications to the knowledge-quiz correlation and frames contributions as preliminary. The central claim, however, is the leading/led-by distinction and its correlation with performance and reliance outcomes. The weakest point in that chain is the unreported reliability and blinding of the coding. The reader identified exactly this. My proposed check would settle it by producing an inter-rater reliability metric and re-testing the headline pattern with blind codes. I do not see an internal inconsistency in the argument, and the qualitative examples (P5/P6 trust-driven led-by, P8 verification failure) are plausible and consistent with prior work. The main issue is external reproducibility, not logical soundness. Therefore a conditional verdict remains appropriate; if the reliability check passes, the paper would be stronger, and if it fails, the claim should be downgraded to a hypothesis-generating observation. No change to the reader's verdict is needed.","tokens_in":8474,"tokens_out":2827,"duration_ms":28798,"concrete_test":"Have two independent coders, blind to holdout F1 scores and to the study's hypotheses, code every conversation turn as leading, led-by, or unclassifiable using only the examples in Section 3.2 plus a written codebook derived from the paper. Compute Cohen's kappa on the full turn set. Then re-run the two headline comparisons with the blind codes: (a) mean F1 for participants coded leading in at least 50% of turns versus the rest, and (b) counts of over-/under-reliance per action type. If kappa is below 0.6, or the F1 gap shrinks to near zero, the central claim should be rephrased as exploratory and dependent on rater interpretation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2 defines Reliance Actions (leading vs. led-by) and says they were 'coded for each conversation turn ... informed by notes and audio transcripts,' but no codebook, annotation guidelines, inter-rater reliability, or blinding procedure is reported. The headline result—'Four of the five top scorers ... were leading ChatGPT in half or more of the conversation turns'—is therefore only as strong as the reproducibility of that coding. Since the researchers who measured holdout F1 also performed the coding, the correlation could be inflated by knowing who succeeded. The same issue affects the second main outcome: Reliance Outcomes (over-/under-reliance) require judgments about whether the LLM was right or wrong, and those judgments are also made without a reported reliability check. This is not a claim of bias; it is a missing validation step for the paper's main independent variable, and n=8 makes the pattern fragile. A workshop paper can still present these as formative observations, but the causal-sounding framing ('participants who ... led-by ... performed worse') needs the coding to be independently reproducible.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper reports a formative study with eight novice ML engineers who debugged a deliberately bugged Random Forest training script on the UCI Adult dataset with free access to ChatGPT. It develops a binary distinction between 'leading' the LLM (specific, planned queries) and being 'led-by' it (open-ended queries followed blindly), codes every conversation turn into these categories and into reliance outcomes (over-, under-, and correct reliance), and relates these to holdout F1 performance and subjective perceptions. The main empirical claim is that participants who led the LLM tended to achieve better task performance—four of the five top scorers led in at least half of the turns—while led-by participants tended to over-rely and perform worse. The paper also identifies meta-cognitive errors (leading wrong hypotheses, poor filtering of broad output, verification difficulties) and proposes augmentations to the novice-LLM interaction paradigm.","tokens_in":8657,"tokens_out":6001,"duration_ms":58209,"significance":"If the leading/led-by taxonomy is reproducible, the paper offers a useful behavioral marker of cognitive engagement in LLM-assisted debugging and connects observable prompting style to objective task outcomes in a low-verifiability domain. The study is appropriately framed as formative, the authors situate the taxonomy in prior paradigms (Barke et al.'s acceleration/exploration, Prather et al.'s shepherding/drifting) and an established reliance framework, and they use an objective holdout F1 score rather than only self-report. The paper also honestly acknowledges the n=8 limitation and does not overstate the quiz correlation. Its value lies mainly in hypothesis generation for future studies of novice-LLM interaction in ML debugging.","major_comments":[{"comment":"The central independent variable of the study is the binary Reliance Action (leading vs. led-by), yet the coding procedure is described only as 'coded for each conversation turn ... informed by notes and audio transcripts,' with no codebook, annotation guidelines, or inter-rater reliability metric reported. Because the same research team that assessed holdout F1 also performed this coding, the headline result—that four of the five top scorers led the LLM in at least half of the turns—could reflect coder expectations rather than a stable behavioral difference. The same concern applies to Reliance Outcomes, which require judgments about whether ChatGPT's advice was correct. Please provide the coding manual, the number of coders and whether they were blind to task performance, and agreement statistics (e.g., Cohen's kappa) for both coding schemes, or explicitly reframe the paper as reporting formative, coder-dependent observations.","section":"Section 3.2"},{"comment":"The claim that 'participants who asked for general guidance (led-by) and shifted the workload to ChatGPT generally performed worse' is stated as a quasi-causal relationship, but the study is observational and the association between leading proportion and F1 is not accompanied by a statistical test or even a per-participant scatterplot. Given that the same section reports a very strong quiz–F1 correlation (r = .93), initial ML knowledge is a plausible confound driving both prompting style and performance. Please report the per-participant values (leading proportion, quiz score, F1 score), use a non-parametric correlation with a clear caveat, and soften the causal phrasing throughout the Discussion.","section":"Section 3.2 (also Figure 1)"},{"comment":"The statement 'Four of the five top scorers ... were leading ChatGPT in half or more of the conversation turns' is presented without a definition of 'top scorers' or a table of per-participant F1 scores, so a reader cannot assess whether this grouping is robust to reasonable alternatives (e.g., using whether each of the three planted errors was fixed rather than F1, or excluding participants whose modifications made performance worse than the unmodified baseline). Please provide the underlying numbers and justify the grouping.","section":"Section 3.2"}],"minor_comments":[{"comment":"The caption says the top bar is reliance outcomes and the bottom bar is actions, while the text says the top bar is reliance actions and the bottom bar is outcomes; please correct the inconsistency.","section":"Figure 1 caption and Section 3.2"},{"comment":"With n = 8, a Pearson r of .93 should be accompanied by a scatter plot and ideally a Spearman correlation, since a single participant could strongly influence the reported value; the current text appropriately de-emphasizes the result, but the figure would make the caveat concrete.","section":"Section 3.1"},{"comment":"The reference format line lists 2018, but the paper is dated 2025; please update the year and venue metadata.","section":"ACM Reference Format"},{"comment":"The label 'Leading Query' for one of the meta-cognitive errors is easily confused with the leading/led-by action categories; consider renaming it (e.g., 'anchoring on a wrong hypothesis') to avoid ambiguity.","section":"Section 3.2"},{"comment":"The five knowledge quiz questions are not included; adding them or providing an OSF/appendix link would improve reproducibility of the quiz measure.","section":"Section 2.2"}],"recommendation":"major_revision","confidential_remarks":"This is a small workshop-scale study, and I would not expect a randomized design or a large sample. However, the main taxonomy is the entire basis for the headline claims, and the lack of reliability evidence and blinding makes the central correlation unverifiable as reported. A revised version with a coding manual, independent reliability check, per-participant data table, and carefully hedged interpretation would be suitable; without these, the contribution remains too fragile to accept."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, quick take on arXiv:2505.08063. It's a 5-page workshop paper with a smart, small study: eight ML novices debug a deliberately broken Random Forest script with ChatGPT. The new bit is applying the known leading/led-by distinction (Barke, Prather) to ML debugging, plus three named meta-cognitive error patterns — injecting wrong leads into queries, being overwhelmed by broad lists, and failing at verification. Those patterns are the most useful part of the paper; they give designers concrete levers.\n\nThe study design is fine for a formative effort: three planted bugs, held-out F1 scoring, a knowledge quiz, think-aloud, interviews. The authors are appropriately cautious about the r=.93 between quiz score and F1 on n=8, and they note the small sample.\n\nThe soft spot is the main independent variable. Reliance actions (leading vs led-by) and reliance outcomes (over/under-rely) were coded from notes and transcripts, but there is no codebook, no second coder, no inter-rater reliability, and no blinding to performance. The same people who scored the code also coded the conversations. That means the headline result — four of five top scorers led the LLM in half or more turns — could partly reflect coder expectations. This is a missing validation step, not evidence of bias. The paper does not acknowledge it. For a workshop paper, that is a common and tolerable gap if the claims stay tentative; here they mostly do, though the abstract and some wording ('participants who ... performed worse') reads a bit more causal than the evidence supports.\n\nBottom line: this is a legitimate formative study that adds useful texture to the novice-LLM interaction literature. It deserves a workshop referee, and I'd be comfortable with it appearing in a workshop venue. If the authors push toward a full archival paper, they need at least a second coder, a pre-registered coding scheme, and ideally a bigger sample. For the reading group, it's a good case study in how small-n qualitative results can be shapeable by coding choices.","headline":"A useful workshop-level study of novice ML debugging with ChatGPT that adds three concrete failure patterns, but the main leading/led-by coding lacks reliability checks, so the headline correlation should be treated as suggestive.","tokens_in":9201,"tokens_out":2023,"would_cite":true,"duration_ms":20176,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"In LLM-assisted ML debugging, who leads the conversation predicts who succeeds.","keywords":["novice-LLM interactions","LLM-assisted coding","debugging","machine learning","mental models","cognitive engagement","over-reliance","appropriate reliance"],"falsifier":"Re-code the eight transcripts with two or more independent raters who are blind to holdout scores, using the paper's definitions of leading and led-by; if inter-rater agreement is low (for example, Cohen's kappa below 0.6), then the reported link between leading behavior and performance may reflect the coders' impressions rather than a stable behavioral difference. A larger pre-registered replication could also test whether the proportion of leading turns predicts holdout $F1$ score, which would settle the claim beyond the current small sample.","tokens_in":8260,"feed_emoji":"🐛","tokens_out":9136,"duration_ms":79572,"temperature":0.7,"pith_summary":"This paper sets out to show that, when novices debug machine-learning code with an LLM assistant, the style of interaction matters more than the amount of help used. In a formative study with eight novice ML engineers given a deliberately broken Random Forest script and free access to ChatGPT, the authors categorize every conversation turn as either 'leading' the LLM (asking specific, planned questions) or being 'led by' the LLM (asking open-ended questions and following suggestions). They report that participants who led the LLM generally achieved higher holdout $F1$ scores, while those who delegated broadly tended to over-rely on ChatGPT; they also observe a strong correlation between a short ML knowledge quiz and task performance. The stakes are educational: if a novice's prompting style predicts both performance and the risk of reinforcing wrong mental models, then tools and tutorials can be designed to keep novices in the leading role.","feed_headline":"Novice ML debuggers who lead ChatGPT outperform those who follow it","feed_subtitle":"In an eight-person debugging study, specific planned questions beat open-ended delegation—and over-reliance follows the led-by pattern.","key_machinery":"The load-bearing object is the turn-level coding scheme that divides every user-ChatGPT exchange into 'leading' (specific, planned, hypothesis-driven questions) versus 'led-by' (open-ended questions followed by uncritical acceptance), together with the four-category reliance outcome scheme (rely on self, rely on LLM, over-rely, under-rely) adapted from prior work on appropriate reliance. This scheme is what connects observable prompt language to task performance: the proportion of leading turns is the predictor, and the outcome categories explain why failures happen, whether from blindly following wrong advice or ignoring correct advice. It also exposes three novice-specific failure modes: injecting a wrong 'lead' into a query, being overwhelmed by long lists of suggestions, and struggling to verify which metrics matter.","core_discovery":"The paper's central claim is that novice-LLM interaction in a high-complexity, low-verifiability task falls into two recognizable patterns, and the pattern predicts the quality of reliance. A 'leading' turn is a specific, planned query aimed at testing the user's own hypothesis, such as asking which hyperparameters matter most for a Random Forest; a 'led-by' turn is an open-ended request such as 'What is wrong with this code?' followed by acceptance of whatever ChatGPT suggests. Coding the eight sessions turn by turn, the authors find that four of the five top scorers on the holdout set led ChatGPT in at least half of their turns, while participants who shifted the workload to ChatGPT performed worse. They also map each turn onto reliance outcomes—rely on self, rely on LLM, over-rely, under-rely—and report that over-reliance clusters in led-by sessions while under-reliance appears when novices drive with a faulty hypothesis, such as asking for feature standardization when it is irrelevant to the model. Finally, they report a correlation of $r = .93$ between the pre-task ML quiz score and holdout $F1$ performance, which they interpret as evidence that domain knowledge, not self-reported experience, governs whether LLM assistance is used well.","pith_inferences":["I infer that the leading/led-by distinction is not specific to ML debugging: any task where answers are hard to verify, such as data analysis or scientific writing, should show the same pattern, and the coding scheme could be tested on transcripts from those domains.","The strong quiz-score correlation, despite the small sample, suggests a cheap practical proxy: a short knowledge test could decide when an LLM assistant should offer structured guidance rather than direct answers.","A testable design extension is to make the LLM explicitly label when it disagrees with the user's stated hypothesis; comparing over-reliance rates and post-task quizzes against a vanilla assistant would show whether correcting faulty mental models reduces the harms observed here.","The paper's framing implies that the 'learning' novices reported was mostly low-level syntax; I infer that measuring mental-model change through a post-task debugging quiz would reveal whether leading behavior also causes learning, not just performance."],"forward_implications":["In LLM-assisted debugging, the share of leading turns can be measured and used to identify novices who are likely to over-rely before they finish the task.","Interventions that push novices to form and state a hypothesis before querying should improve both debugging outcomes and the accuracy of reliance, since leading behavior is associated with better holdout performance.","Domain knowledge, as measured by a short quiz, predicts performance with an LLM assistant better than self-reported ML experience, so assessment tools should measure actual knowledge rather than confidence.","LLM interfaces for novices need to handle two distinct failure modes: they should push back on incorrect user 'leads' to prevent over-reliance, and they should help filter broad responses to prevent under-reliance from cognitive overload.","Because participants said they would prompt differently in hindsight, brief prompting tutorials may shift led-by novices toward leading behavior."],"supporting_citations":[{"why":"Defines the acceleration-versus-exploration interaction styles that the leading/led-by coding adapts to ML debugging.","marker":"[5]"},{"why":"Documents shepherding vs drifting behavior in novice use of code-generating LLMs, the pattern behind the led-by category.","marker":"[21]"},{"why":"Provides the appropriate-reliance evaluation approach used to classify reliance outcomes in this study.","marker":"[7]"},{"why":"Conceptualizes appropriate reliance on AI advice and supplies the over-reliance and under-reliance outcome definitions.","marker":"[23]"},{"why":"Supplies the Adult Income dataset used to build the buggy ML debugging task.","marker":"[6]"},{"why":"Earlier evidence that novices over-rely on LLM code generators; motivates the research question.","marker":"[15]"},{"why":"Argues that ML systems are complex and difficult to verify, justifying the choice of ML debugging as the task.","marker":"[1]"}],"fun_headline_variants":["Leading ChatGPT, not following it, predicts novice ML debugging success","In ML debugging, novices who plan prompts outperform those who delegate","Over-reliance on ChatGPT: led-by pattern, not leading, is the trap","Specific queries beat 'what's wrong?' for novice ML debugging with ChatGPT","ChatGPT help: novices who lead with hypotheses debug better than followers"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's central pattern rests on the authors' binary coding of every conversation turn as leading or led-by, and the paper reports no check that independent raters would apply that coding the same way.","fun_headline_variants_meta":{"raw":{"variants":["Leading ChatGPT, not following it, predicts novice ML debugging success","In ML debugging, novices who plan prompts outperform those who delegate","Over-reliance on ChatGPT: led-by pattern, not leading, is the trap","Specific queries beat 'what's wrong?' for novice ML debugging with ChatGPT","ChatGPT help: novices who lead with hypotheses debug better than followers"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000869,"raw_usage":{"total_tokens":3797,"prompt_tokens":1007,"completion_tokens":2790,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":623,"completion_tokens_details":{"reasoning_tokens":2695}},"tokens_in":623,"tokens_out":2790,"duration_ms":21296,"temperature":1.0,"reasoning_tokens":2695,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:04:22.821041+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-code the eight transcripts with two or more independent raters who are blind to holdout scores, using the paper's definitions of leading and led-by; if inter-rater agreement is low (for example, Cohen's kappa below 0.6), then the reported link between leading behavior and performance may reflect the coders' impressions rather than a stable behavioral difference. A larger pre-registered replication could also test whether the proportion of leading turns predicts holdout $F1$ score, which would settle the claim beyond the current small sample.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Argues that ML systems are complex and difficult to verify, justifying the choice of ML debugging as the task."}],"review_version":1}