On Impact of Semantically Similar Apps in Android Malware Datasets

Malware authors often reuse program segments from other applications (apps) to perform similar kind of malicious activities, such as information stealing and sending SMS to premium rate numbers. Consequently, multiple semantically similar malware samples may exist within a malware family or dataset. The presence of such apps, unbeknownst to many researchers, may inadvertently influence the evaluation of their Machine Learning (ML) models. In this paper, we investigate the impact of semantically similar apps on the performance measures of ML-based Android malware detectors. Through experiments on the Drebin dataset, we assessed the performance of distinct ML models based on opcode, permissions and API call features of both malware and goodware applications, with and without semantically similar apps. Our findings show that after removing exact duplicate apps from the dataset, the malware detection rate (True Positive Rate) of opcode-based ML models decreased from 0.94 to 0.85, permission-based ML models decreased from 0.94 to 0.90 and API call-based ML models decreased from 0.95 to 0.91. To address this issue, we propose the use of the Euclidean distance metric to identify and eliminate the similar features before evaluating malware detection mechanisms. Implementing this recommendation can enhance the accuracy and reliability of ML-based Android malware detectors.