The reviewed record of science sign in
Pith

arxiv: 2306.09541 · v3 · pith:NSN27FFQ · submitted 2023-06-15 · cs.HC · cs.PL

Validating AI-Generated Code with Live Programming

Kasra Ferdowsi , Ruanqianqian Huang , Michael B. James , Nadia Polikarpova , Sorin Lerner This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:NSN27FFQrecord.jsonopen to challenge →

classification cs.HC cs.PL
keywords programmingai-generatedai-poweredchallengecodedevelopersenvironmentlive
0
0 comments X
read the original abstract

AI-powered programming assistants are increasingly gaining popularity, with GitHub Copilot alone used by over a million developers worldwide. These tools are far from perfect, however, producing code suggestions that may be incorrect in subtle ways. As a result, developers face a new challenge: validating AI's suggestions. This paper explores whether Live Programming (LP), a continuous display of a program's runtime values, can help address this challenge. To answer this question, we built a Python editor that combines an AI-powered programming assistant with an existing LP environment. Using this environment in a between-subjects study (N=17), we found that by lowering the cost of validation by execution, LP can mitigate over- and under-reliance on AI-generated programs and reduce the cognitive load of validation for certain types of tasks.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Understanding the Human-LLM Dynamic: A Literature Survey of LLM Use in Programming Tasks

    cs.SE 2024-10 unverdicted novelty 3.0

    A survey of user studies on LLM use in programming that identifies interaction behaviors, mixed benefits and weaknesses, and factors influencing human and task performance.