A Resource-Constrained and Privacy-Preserving Edge-Computing-Enabled Clinical Decision System: A Federated Reinforcement Learning Approach

Internet-of-Things-enabled E-health system, which could monitor and collect the personal health information (PHI), has gradually transformed the clinical treatment to a more personalized way with in-home monitoring smart devices. Then, with the collected PHI, clinical decision support systems (CDSSs), which are based on data mining techniques and historical electronic medical records (EMRs) to help clinicians make proper treatment decisions, have attracted considerable attention. To address issues, such as network congestion and low rate of responsiveness for traditional methods when implementing CDSSs, we integrate the technologies mobile-edge computing (MEC) and software-defined networking for exploiting the computation resources and storage capacities among edge nodes (ENs) (i.e., MEC servers) in our model. Based on this integrated system, each edge node will deploy a double deep Q-network (DDQN) to obtain a stable and sequential clinical treatment policy. It is enabled by a novel fully decentralized federated framework (FDFF) for aggregating models of DDQN and extracting the knowledge from EMRs across all ENs. Furthermore, we discuss the convergence of FDFF in resource-constrained environments. However, since most EMRs are faced with stringent privacy concerns, we adopt two additively homomorphic encryption schemes to prevent leakage of EMRs' privacy during the training process of FDFF. Finally, we measure the time cost of our additively homomorphic encryption schemes and validate DDQN with experiments on large data sets based on FDFF, which shows promising performance on clinician treatment.