Multi-Level Generative Pretrained Transformer for Improving Malware Detection Performance

Generating diverse and realistic malware variants is critical to improve the performance of deep learning-based Malware Detection Systems (MDSs) and fight against an increasing number of complex malware attacks. Traditionally, researchers construct the single generative model based on the whole dataset, which may suffer from mode collapse and scalability problem. In order to overcome these problems, we propose Multi-level Generative Pretrained Transformer (MLGPT) which organizes multiple GPTs in the tree. Each GPT in the tree can learn the unique pattern of malware language for one malware subfamily. Consequently, MLGPT has great potentials to produce more diverse and realistic malware variants than the single generative model. Experimental results show that performance improvement of MLGPT is statistically significant as compared to the single generative model while the construction time of MLGPT is comparable to the single generative model due to the parallel strategy.