Mixup Based Privacy Preserving Mixed Collaboration Learning

The amount of high-quality data determines the performance of the deep learning model. In reality, the data is often physically distributed in different organizations, and model averaging can train a deep model on the distributed data, while providing competitive performance compared with training a model on the centralized data. However, it cannot prevent inversion attack, as the intermediate parameters are transmitted during training. Some data enhancement methods, such as mixup, can effectively enhance the data privacy. In this paper, we propose a novel model averaging method combined with mixup, which provides protection against inversion attack. Besides we conduct experiments using state-of-the-art deep network architectures on multiple types of dataset to show that our method improves the classification accuracy of models.

I. Introduction

In recent years, deep learning has attracted great attention from industry and academia, and has been greatly developed in the medical, financial, education, Internet, and etc [1]–[3]. The performance of the deep neural network model is closely related to the scale and quality of the data [4]. More high-quality data can greatly improve the performance of the model. In reality, data is often held by different organizations. For example, for diabetic retinopathy, different hospitals can collect case samples of different characteristics (such as region and age). If these samples are put together, the accuracy and robustness of the detection model can be significantly improved. However, privacy and data laws in some countries or organizations prohibit transmission of raw data across country or organization [5], [6]. In view of the current situation that data in various professional fields is not open, how to make full use of richer data features under the premise of ensuring data privacy of different organizations has become an urgent challenge for deep learning.