CyberLlama2 - MEDICALHARM Threat Modeling Assistant

Threat Modeling, a shift-left cybersecurity activity to build security into a software design, has become a challenge for many industries, including Modern Medical Devices. With the advancement of Large Language Models (LLM), more industries are adopting this phenomenon to understand and solve domain-specific challenges. However, thus far, little research has evaluated the effectiveness of LLM in solving threat modeling challenges. To alleviate this problem, we developed a threat modeling assisted LLM to assist MEDICALHARM in identifying cybersecurity, privacy, and safety threats in the Modern Medical Device space. We developed a specialized decoder-only model, CyberLlama2, to assist threat modeling using a large set of -146k- cybersecurity instructions to fine-tune Llama2. The results show an improved performance of our proposed CyberLlama2 model over the baseline and other cybersecurity models.

I. Introduction

Artificial Intelligence (AI) has significantly advanced across various industries, fields, and domains in recent years. These advancements include manufacturing [1] and the healthcare sectors [2]. Fine-tuning is a commonly used technique to obtain a domain-specific LLM with related data and has been shown to optimize the accuracy of input queries and the coherence of the models [2]. Various advances in research on domain-specific LLMs, including cybersecurity, have been noted in recent studies [3], [4]. However, current cybersecurity-specific LLMs are usually fine-tuned to general cybersecurity concepts and are not customized to assist threat modeling methodologies, let alone threat modeling that requires specific knowledge of general threats and vulnerabilities. This is particularly challenging when considering the complexities of medical devices, which involve intrinsic components such as various sensors, actuators, and unique safety concerns for patient health. Thus, we fine-tune Llama2 [5] LLM with well-constructed cybersecurity, privacy, and safety instructions to adopt MEDICALHARM threat modeling.