Shadows can be Dangerous: Stealthy and Effective Physical-world Adversarial Attack by Natural Phenomenon

Estimating the risk level of adversarial examples is essential for safely deploying machine learning models in the real world. One popular approach for physical-world attacks is to adopt the “sticker-pasting” strategy, which however suffers from some limitations, including difficulties in access to the target or printing by valid colors. A new type of non-invasive attacks emerged recently, which attempt to cast perturbation onto the target by optics based tools, such as laser beam and projector. However, the added optical patterns are artificial but not natural. Thus, they are still conspicuous and attention-grabbed, and can be easily noticed by humans. In this paper, we study a new type of optical adversarial examples, in which the perturbations are generated by a very common natural phenomenon, shadow, to achieve naturalistic and stealthy physical-world adversarial attack under the black-box setting. We extensively evaluate the effectiveness of this new attack on both simulated and real-world environments. Experimental results on traffic sign recognition demonstrate that our algorithm can generate adversarial examples effectively, reaching 98.23% and 90.47% success rates on LISA and GTSRB test sets respectively, while continuously misleading a moving camera over 95% of the time in real-world scenarios. We also offer discussions about the limitations and the defense mechanism of this attack¹¹Our code is available at https://github.com/hncszyq/ShadowAttack.

1. Introduction

In the past few years, we have witnessed the great success of deep neural networks (DNNs) in a variety of computer vision tasks, such as image classification, object detection, scene segmentation and so on. However, recent studies have revealed that DNNs based models are vulnerable to adversarial examples, even though the added magnitude of perturbations is small. In safety sensitive scenarios, such as autonomous driving [18] and medical diagnosis [19], adversarial inputs would enforce a machine learning system to produce erroneous decision, leading to unexpected situations that may be potentially dangerous.