Deep Q-Network-Based Open-Set Intrusion Detection Solution for Industrial Internet of Things

Industrial Internet of Things (IIoT) has brought a lot of convenience for the industrial world to digitization, automation, and intelligence, but it inevitably introduces inherent cyber security risks, resulting in an issue that traditional intrusion detection techniques are no longer sufficient for IIoT environments. To solve this issue, we propose an open-set solution called a DC-intrusion detection system (IDS) for IIoT based on deep reinforcement learning. In this solution, the open-set recognition problem in intrusion detection is modeled as a discrete-time Markov decision process, and a deep
$Q$
-network (DQN) is employed to solve it. Meanwhile, a conditional variational autoencoder is introduced to the value network in DQN. Therefore, the open-set recognition problem in intrusion detection is divided into two subproblems, namely, known traffic fine-grained classification problem and unknown attacks recognition problem. We use DQN to solve the known traffic fine-grained classification problem. Since the reconstruction error of known traffic is generally smaller than the reconstruction error of unknown attacks, we use reconstruction error to recognize unknown attacks. Experiments on the IIoT data set TON-IoT demonstrate the effectiveness of the DC-IDS model, which achieves better performance in terms of the recognition rate of unknown attacks as well as the stability of the model compared to previous proposed methods.