Software Defect Prediction via Code Language Models

Software defect prediction has been effective practice to improve software quality and avoid attacks. Traditional defect prediction models with static or other features cannot understand syntactic and semantic structures well. Recently, large language models like GPT-series show impressive energy on considerable number of tasks. In the realm of software engineering, it has been shown that code language models largely benefit a broad set of code-related downstream tasks. In this paper, we employ code language models with distinct architectures for defect prediction. we propose a unified bi-modal input representation to enhance the comprehension of semantic information. Furthermore, a new bi-modal dataset is proposed based on the PROMISE repository and the engineering files. A large number of experiments are conducted on both within and cross project for evaluation and comparison. The results demonstrate the effectiveness of our proposed method for software defect prediction, which outperforms the state-of-the-art baselines. Among the three architectures, encoder-only achieve the most effective and efficient.