Bullet-Screen-Emoji Attack With Temporal Difference Noise for Video Action Recognition

Recent studies have shown that video action recognition models are also vulnerable to fooling by adversarial samples. However, currently existing video attack methods usually require high computational overhead (e.g., they generate adversarial perturbations for all frames by default), and most of them are difficult to implement printable attacks in the physical world. To address the above issues, we devise a novel efficient and effective framework for video action recognition attack: Bullet-Screen-Emoji Attack with Temporal Difference Noise (BSE), a reinforcement learning-based black-box attack method that fools the model by simply generating adversarial bullet screens for key frame and scrolling them on clean video. The agent is optimized to make the optimal actions, i.e., searching key frame. Moreover, we introduce a simple and effective temporal difference noise to enhance the attack capability of the adversarial bullet screen and accelerate the convergence speed. Most importantly, BSE enables printable physical attacks. Extensive experiments show that our proposed BSE achieves promising attack performance on mainstream datasets (HMDB51, UCF101 and Kinetics-400) and in the physical world with high efficiency.

I. Introduction

With the development of Deep Neural Networks, adversarial attack and defense are gradually becoming a hot research direction [1], [2], [3], [4], [5]. Based on whether the gradient and structure information of the target model can be accessed during the attack, adversarial attacks are classified into white-box attack and black-box attack [6]. White-box attack has been very successful, with robust performance and almost real-time efficiency in handling attacks in the digital world, but they are not applicable to real-world application scenarios, because in real-world application scenarios, the internal structure and parameters such as gradient of target model are generally not directly accessible to the attacker. Black-box attacks are more in line with real-world application scenarios, which use query-based methods to launch attacks on the model and adjust the attack strategy based on the model’s predicted output. With the development of adversarial attack, black-box attack has made great breakthroughs [7], [8], [9].