A Safe Transfer Learning Approach Inspired by Desensitization Therapy

Transfer learning is a mainstream method to solve data scarcity tasks, but it faces the threat of defect inheritance, which makes the student model vulnerable to adversarial example attacks. Traditional solutions include model slicing and model reconstruction. However, due to the uninterpretability of deep neural networks, model slicing methods often adopt a conservative way, which leads to the limited resistance of the model to adversarial example attacks. Inspired by the theory of the biomedical field, this paper proposes a node optimization method based on desensitization therapy (STL-DMP). In STL-DMP, the student model is regarded as an allergic person, the abnormal state of the student model after being attacked by the adversarial sample is regarded as an allergic reaction, the interference is regarded as an allergen, model weights are regarded as sensitizing-cell activity mediators, and the loss function is regarded as an immune regulatory signal. Based on the above statement, STL-DMP uses interference (allergens) to prepare adversarial samples (vaccines). The student model (allergy) is exposed to a specific perturbation (allergen) by training (treatment) with dedicated adversarial examples (vaccines). The model weights (sensitizing-cell activity mediators) are gradually weakened (consumed to an equilibrium value), thus eliminating the threat of adversarial sample attacks (sensitized state). Experiments show that STL-DMP improves the resistance of the model to adversarial attacks while maintaining accuracy.