A Taxonomy of Inefficiencies in LLM-Generated Python Code
read the original abstract
Large Language Models (LLMs) are widely adopted for automated code generation with promising results. Although prior research has assessed LLM-generated code and identified various quality issues -- such as redundancy, poor maintainability, and sub-optimal performance a systematic understanding and categorization of these inefficiencies remain unexplored. Without such knowledge, practitioners struggle to optimize LLM-generated code for real-world applications, limiting its adoption. This study can also guide improving code LLMs, enhancing the quality and efficiency of code generation. Therefore, in this study, we empirically investigate inefficiencies in LLM-generated code by state-of-the-art models, i.e., CodeLlama, DeepSeek-Coder, and CodeGemma. To do so, we analyze 492 generated code snippets in the HumanEval++ dataset. We then construct a taxonomy of inefficiencies in LLM-generated code that includes 5 categories General Logic, Performance, Readability, Maintainability, and Errors) and 19 subcategories of inefficiencies. We then validate the proposed taxonomy through an online survey with 58 LLM practitioners and researchers. Our study indicates that logic and performance-related inefficiencies are the most popular, relevant, and frequently co-occur and impact overall code quality inefficiency. Our taxonomy provides a structured basis for evaluating the quality LLM-generated code and guiding future research to improve code generation efficiency.
This paper has not been read by Pith yet.
Forward citations
Cited by 5 Pith papers
-
ARIADNE: Agentic Reward-Informed Adaptive Decision Exploration via Blackboard-Driven MCTS for Competitive Program Generation
ARIADNE combines blackboard architecture with MCTS to coordinate strategy, code, test, evaluation, and repair stages, yielding higher Pass@1 scores than prior LLM baselines on APPS, CodeContests, and related benchmarks.
-
SpecDB: LLM-Generated Customized Databases via Feature-Oriented Decomposition
SpecDB generates a 23,779-line Rust database via LLM subagents that matches PostgreSQL and MySQL tpmC on TPC-C while using roughly 3% of their code size.
-
Bridging Generation and Training: A Systematic Review of Quality Issues in LLMs for Code
A review of 114 studies creates taxonomies for code and data quality issues, formalizes 18 propagation mechanisms from training data defects to LLM-generated code defects, and synthesizes detection and mitigation techniques.
-
An Empirical Study on Logging Evolution On Stack Overflow: Trends, Topics, and Challenges
Empirical study of Stack Overflow logging posts identifies 11 topics where containerized environments show the highest difficulty via unanswered rates and resolution time.
-
Architectures of Error: A Philosophical Inquiry into AI and Human Code Generation
Develops a conceptual distinction between human-cognitive and artificial-stochastic error architectures in code generation, drawing on Dennett, Rescher, and Floridi to explore implications for AI-human collaboration.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.