A Novel Confidence Score Exploiting Attacks on Classification Trees and Random Forest Classifiers

The need for studies on the privacy risks of machine learning models has been increasing as using sensitive data in training them has become prevalent in real-world applications. Decision tree and random forest models have been used for data mining for several decades. Yet, there are not enough studies on the privacy risks of these models. In this paper, we present two novel attribute inference attacks, i.e., CTAIA and RFAIA, for the decision tree and random forest classifiers, respectively. CTAIA is a black-box attack, and RFAIA is a white-box attack. Our attacks utilize the confidence score information from the model outputs in a novel way. Specifically, our attacks use the zero values in confidence scores of the decision tree classifiers. A zero confidence score for a specific class means that there is no training data sample for that class. This fact, the embedding of the information about the number of the train data samples in the confidence score, is used to exclude the candidate values of the target attribute. We define the train data samples, which an attacker of an attribute inference attack can infer the values of the sensitive attribute with 100% confidence, as "high-risk" data records. For the decision tree classifiers or classification trees, CTAIA selects some data records and infers the values of the target attribute of them with 100% accuracy, making them "high-risk" data records. Similarly, for the random forest classifiers, RFAIA selects some data records. Depending on whether the bootstrap sampling is used in training the classifiers or not, the RFAIA has 100% or near 100% attack accuracy for the selected data records. Therefore, in the case of random forest classifiers, for simplicity, we loosen the above-mentioned definition of the "high-risk" data records and call the data records selected by RFAIA the "high-risk" data in this paper. We have experimentally shown the effectiveness of our attack using three public datasets.