A Self-Iteration Code Generation Method Based on Large Language Models

Although large language models (LLMs) have demonstrated impressive performance in code generation, they still face challenges when dealing with complex code generation tasks. In the software development process, humans often refine complex tasks iteratively and continuously modify and improve them. Inspired by this, we propose a self-iteration code generation framework based on large language models like ChatGPT. To realize this idea, we incorporated software development methodologies into the self-iteration framework. We introduced four roles into each cycle, including analyst, designer, developer, and tester. Each role performs different tasks during the self-iteration cycle, with analyst and designer continuously improving requirements analysis and task design based on the testing feedback provided by tester, while developer are responsible for refactoring or modifying code until it passes testing, concluding the entire self-iteration process. We conducted extensive experiments on multiple benchmarks. The experimental results indicate: (1) The code generated by the self-iteration framework achieves up to a 21.3% relative improvement in Pass@1 compared to direct code generation. (2) The self-iteration framework also exhibits strong generalization performance, enhancing code generation quality for different large language models."Failure is simply the opportunity to begin again, this time more intelligently."- Henry Ford

I. Introduction

As one of the most important research areas in programming theory, code generation refers to the process of transforming specifications or descriptions at a high-level abstraction into executable computer program code using automated tools or programming techniques. In recent years, code generation has made remarkable progress in both the academic and industrial sectors [1]–[3]. It’s worth noting that large language models (LLMs) have demonstrated impressive capabilities in the field of code generation [4]–[6].