A novel backdoor scenario target the vulnerability of Prompt-as-a-Service for code intelligence models

With the rapid development of prompt tuning technologies and the emergence of prompt-as-a-service platforms, the security of prompt service has attracted the attention of researchers. Deep neural networks face susceptibility to various adversaries. A particularly malicious altering model behavior is backdoor attack, where model predictions exhibit divergence in the presence of specific triggers in inputs. We concentrate on backdoor attacks in code intelligence models. In this paper, we aim to propose a novel backdoor attack scenario target the security of prompt service. In traditional backdoor scenario, the victim model is fine-tuned on the downstream poisoned dataset to establish a shortcut between the trigger and the target label. This scenario suffers from flaws such as rare words inserted into code snippets resulting in alterations to the code’ s semantics. And only those attackers who are familiar with specific triggers can launch an attack. Instead of introducing extra rare words to code snippets, our attack scenario choose to employ prompt itself as the triggers. We evaluate our methods on three popular code intelligence tasks, including code defect detection, clone detection and code summarization. Experimental results indicate that our methods can achieve an attack success rate of 95% to 99% with almost no sacrifice in accuracy on the original task, exhibiting an average performance degradation of less than 1%. We hope exposed potential security risks hidden in prompt service can raise awareness among researchers.

I. Introduction

The rapid development of Deep Neural Networks (DNN) coupled with the emergence of Open-Source Software (OSS) data has led to the accelerated advancement of DNN-based code intelligence models [1]. For example, Feng et.al [2] proposed a pre-trained code model CodeBERT with an Encoder architecture. The model is pre-trained on a large-scale code corpus to learn the semantic information of code snippets, providing powerful representations for tasks related to source code. To facilitate code generation tasks, Wang et.al [3] proposed a sequence-to-sequence pre-trained model CodeT5. At the same time, different kinds of code intelligence tasks are emerging, such as code defect detection [4], code summary generation [5] and so on. It has already become a common paradigm to fine-tune pre-trained models on downstream datasets to improve their performance on specific tasks [6].