Model Inversion Attacks for Prediction Systems: Without Knowledge of Non-Sensitive Attributes

While online services based on machine learning (ML) have been attracting considerable attention in both academic and business, privacy issues are becoming a threat that cannot be ignored. Recently, Fredrikson et al. [USENIX 2014] proposed a new paradigm of model inversion attacks, which allows an adversary to expose the sensitive information of users by using an ML system for an unintended purpose. In particular, the attack reveals the sensitive attribute values of the target user by using their non-sensitive attributes and the output of the ML model. Here, for the attack to succeed, the adversary needs to possess the non-sensitive attribute values of the target user prior to the attack. However, in reality, even if this information (i.e., non-sensitive attributes) is not necessarily information the user regards as sensitive, it may be difficult for the adversary to actually acquire it. In this paper, we propose a general model inversion (GMI) framework to capture the above scenario where knowledge of the non-sensitive attributes is not necessarily provided. Here, our framework also captures the scenario of Fredrikson et al. Notably, we generalize the paradigm of Fredrikson et al. by additionally modeling the amount of auxiliary information the adversary possesses at the time of the attack. Our proposed GMI framework enables a new type of model inversion attack for prediction systems, which can be carried out without knowledge of the non-sensitive attributes. At a high level, we use the paradigm of data poisoning in a novel way and inject malicious data into the set of training data to modify the ML model into a target ML model, which we can attack without having to have knowledge of the non-sensitive attributes. Our new attack enables the inference of sensitive attributes in the user input from only the output of the ML model, even when the non-sensitive attributes of the user are not available to the adversary. Finally, we provide a concrete algorithm of our model inversion attack on prediction systems based on linear regression models, and give a detailed description of how the data poisoning algorithm is constructed.We evaluate the performance of our new model inversion attack without the knowledge of non-sensitive attributes through experiments with actual data sets.

1. Introduction

The artificial intelligence (AI) paradigm is rapidly gaining momentum and machine learning (ML) has been playing a core role in modern IT services (e.g., product recommendation [1]–[3], personalized medicine [4]–[6], destination prediction [7]–[9]). In many of these services, ML models make some predictions using the personal data of users as input. Since the personal data often include sensitive information, which the users may want to keep confidential (e.g., purchase logs of sensitive items, genetic markers, home addresses), the handling of personal data in the ML model is raising concerns in relation to privacy issues. To this extent, analyzing the vulnerability (or security) of ML systems has attracted much active research.