REVIEW 13 cited by
SOEN-101: Code Generation by Emulating Software Process Models Using Large Language Model Agents
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Software process models are essential to facilitate collaboration and communication among software teams to solve complex development tasks. Inspired by these software engineering practices, we present FlowGen - a code generation framework that emulates software process models based on multiple Large Language Model (LLM) agents. We emulate three process models, FlowGenWaterfall, FlowGenTDD, and FlowGenScrum, by assigning LLM agents to embody roles (i.e., requirement engineer, architect, developer, tester, and scrum master) that correspond to everyday development activities and organize their communication patterns. The agents work collaboratively using chain-of-thought and prompt composition with continuous self-refinement to improve the code quality. We use GPT3.5 as our underlying LLM and several baselines (RawGPT, CodeT, Reflexion) to evaluate code generation on four benchmarks: HumanEval, HumanEval-ET, MBPP, and MBPP-ET. Our findings show that FlowGenScrum excels compared to other process models, achieving a Pass@1 of 75.2, 65.5, 82.5, and 56.7 in HumanEval, HumanEval-ET, MBPP, and MBPP-ET, respectively (an average of 15% improvement over RawGPT). Compared with other state-of-the-art techniques, FlowGenScrum achieves a higher Pass@1 in MBPP compared to CodeT, with both outperforming Reflexion. Notably, integrating CodeT into FlowGenScrum resulted in statistically significant improvements, achieving the highest Pass@1 scores. Our analysis also reveals that the development activities impacted code smell and exception handling differently, with design and code review adding more exception handling and reducing code smells. Finally, FlowGen models maintain stable Pass@1 scores across GPT3.5 versions and temperature values, highlighting the effectiveness of software process models in enhancing the quality and stability of LLM-generated code.
Forward citations
Cited by 13 Pith papers
-
FVRuleLearner: Operator-Level Reasoning Tree (Op-Tree)-Based Rules Learning for Formal Verification
FVRuleLearner retrieves learned operator-level reasoning rules to boost the functional correctness of LLM-generated SystemVerilog assertions by roughly 30 percentage points over simple prompting baselines.
-
Bridging Behavior and Implementation: Automated Java Glue Code Generation for Behavior-Driven Development
AutoGlue, a behavior-first multi-agent LLM framework, generates Java BDD glue code that matches project APIs, with 46.1% of 1,307 outputs rated directly usable.
-
WebDesignIter: Co-Evolving Design Knowledge for Repository-Level Front-End Code Generation
A co-evolving design-and-code knowledge graph improves multi-iteration front-end repository generation by ~9.5 Pass@2 points and cuts input tokens 25–30× versus agent baselines.
-
Can Large Language Models Integrate Spatial Data? Empirical Insights into Reasoning Strengths and Computational Weaknesses
LLMs only become competitive at spatial data integration when given pre-computed geometric features; a review-and-refine prompt then exceeds hand-tuned heuristics.
-
A Hierarchical and Evolvable Benchmark for Fine-Grained Code Instruction Following with Multi-Turn Feedback
MultiCodeIF introduces a 2,021-task, 14-language benchmark with 27 constraint types to evaluate code instruction following, finding that multi-level constraints sharply reduce model success and iterative feedback subs...
-
Recognition Without Mitigation: Ethical Frameworks in Autonomous Offensive-LLM Agent Research
In offensive-LLM agent papers, dual-use risk is acknowledged in 39% of papers but concrete mitigations appear in only 7%.
-
JARVIS: A Multi-Agent Code Assistant for High-Quality EDA Script Generation
A multi-agent LLM framework with rule enforcement, compiler feedback, and retrieval achieves 92/93/81% pass@1 on three self-built EDA benchmarks, up from 67/62/43% for the best single model.
-
A Causal Perspective on Measuring, Explaining and Mitigating Smells in LLM-Generated Code
A probabilistic score of code-smell propensity in LLM output is validated, used in a causal analysis, and shown to drop when prompts explicitly discourage known smells.
-
Generating Privacy Stories From Software Documentation
LLMs can extract privacy behaviors from software documents and draft privacy stories, but the best overall F1 is 0.766, not the abstract's 0.8+.
-
GPTFootprint: Increasing Consumer Awareness of the Environmental Impacts of LLMs
An eco-feedback browser extension for ChatGPT raises user awareness of energy and water use, but a nine-participant study finds limited effect on query frequency.
-
WALL: A Web Application for Automated Quality Assurance using Large Language Models
WALL combines SonarQube with GPT-3.5 Turbo and GPT-4o to automatically fix detected code issues, reporting 71.6% and 85.5% revision rates plus cost reductions from a hybrid workflow.
-
Cognitive Agents Powered by Large Language Models for Agile Software Project Management
LLM agents acting as Agile roles produced plausible project artifacts in simulation, but the claimed improvements over human teams are unsupported because no comparison or validated metrics are provided.
-
In-Context Learning as an Effective Estimator of Functional Correctness of LLM-Generated Code
Few-shot in-context examples improve LLM-based functional correctness estimation for generated code relative to zero-shot judgment, but the gains are modest and uneven.
Discussion (0). Sign in to comment.