DA2F: Research on Robustness of Deep Learning Models Using Approximate Activation Function | IEEE Conference Publication | IEEE Xplore

DA2F: Research on Robustness of Deep Learning Models Using Approximate Activation Function


Abstract:

Deep learning structures have been adopted in many application domains. However, these architectures are vulnerable to adversarial attacks which are often instantiated by...Show More

Abstract:

Deep learning structures have been adopted in many application domains. However, these architectures are vulnerable to adversarial attacks which are often instantiated by adversarial examples: carefully crafted inputs by adding disturbances that are imperceptible to humans can easily mislead a learned classifier to make incorrect predictions. Since deep learning is fast achieving the maturity to enter into safety- critical and security-sensitive applications, such attacks may have catastrophic security and safety consequences. In this paper, we propose a new software-based approach to enhance the robustness of deep learning models to adversarial attacks which we call the Defensive Approximate Activation Function (DA 2F). Specifically, we mainly consider deep learning architectures using the sigmoid function or tanh function, two complex non-linear involving exponential operation functions, as activation functions. We propose the piecewise linear approximation method where a new non-uniform segmentation scheme is presented. By replacing the exact activation function used in the deep learning architectures with the approximate activation function, which reduces computation cost theoretically and improves deep learning models' robustness to adversarial samples. The experiments validated that our approach was effective in defending against adversarial attacks. For LeNet-5 CNN architecture along with MNIST datasets, the approximate classifiers were more robust against adversarial attacks than the exact classifiers, with negligible loss in accuracy.
Date of Conference: 18-20 December 2022
Date Added to IEEE Xplore: 28 March 2023
ISBN Information:
Conference Location: Hainan, China

Funding Agency:


I. Introduction

Deep learning (DL) at the heart of the current rise of artifi-cial intelligence has been widely adopted in many application domains to solve a wide range of real-life problems. With the rapid progress in developing and deploying DL models, DL is fast achieving the maturity to enter into safety-critical and security-sensitive applications, such as autonomous driving [1], surveillance [2], malware detection [3], robotics [4], and speech recognition [5]. However, Szegedy [6] discovered in the image field that machine learning (ML) models are vulnerable to adversarial attacks. Such attacks are often instantiated by adversarial examples: carefully crafted inputs by adding disturbances that are imperceptible to humans can easily mislead a learned classifier to make incorrect predictions, which may cause catastrophic security and safety consequences.

References

References is not available for this document.