A Resource-Constrained and Privacy-Preserving Edge-Computing-Enabled Clinical Decision System: A Federated Reinforcement Learning Approach

Internet-of-Things-enabled E-health system, which could monitor and collect the personal health information (PHI), has gradually transformed the clinical treatment to a more personalized way with in-home monitoring smart devices. Then, with the collected PHI, clinical decision support systems (CDSSs), which are based on data mining techniques and historical electronic medical records (EMRs) to help clinicians make proper treatment decisions, have attracted considerable attention. To address issues, such as network congestion and low rate of responsiveness for traditional methods when implementing CDSSs, we integrate the technologies mobile-edge computing (MEC) and software-defined networking for exploiting the computation resources and storage capacities among edge nodes (ENs) (i.e., MEC servers) in our model. Based on this integrated system, each edge node will deploy a double deep Q-network (DDQN) to obtain a stable and sequential clinical treatment policy. It is enabled by a novel fully decentralized federated framework (FDFF) for aggregating models of DDQN and extracting the knowledge from EMRs across all ENs. Furthermore, we discuss the convergence of FDFF in resource-constrained environments. However, since most EMRs are faced with stringent privacy concerns, we adopt two additively homomorphic encryption schemes to prevent leakage of EMRs' privacy during the training process of FDFF. Finally, we measure the time cost of our additively homomorphic encryption schemes and validate DDQN with experiments on large data sets based on FDFF, which shows promising performance on clinician treatment.

I. Introduction

Internet-of-Things (IoT)-enabled E-health (IoT-Ehealth) system, which refers to a variety of devices that monitor and collect the personal health information (PHI) [1], has gradually transformed the traditional clinical treatment to a more personalized way by incorporating information technology. Consequently, with the help of collected PHI, clinical decision support systems (CDSSs) [2] based on data mining techniques and historical electronic medical records (EMRs) are playing a prior role in maintaining health for individuals, since it could reduce hospitalization, achieve real-time feedback, and give information to clinicians for making better clinical decisions [3]. However, as many patients are living in remote areas, traditional CDSSs enabled by cloud computing cannot provide a sufficiently high rate of responsiveness and scalability, and it will impose an exceedingly heavy traffic load to communication networks [4]. To address these issues, mobile-edge computing (MEC) [5] has emerged as an ideal computing platform for clinical decisions in the era of IoT.