Class-aware Variational Auto-encoder for Open Set Recognition

Compared with traditional classification models trained under the closed world assumption, Open Set Recognition (OSR) requires accurate classification for known classes as well as rejection for unknown ones. By modeling the distribution of each known class, Conditional Variational Auto-encoder (CVAE) has achieved great success in OSR, even though it was originally proposed for image generation. In this paper, we propose a novel two-stage learning framework, Class-aware Variational Auto-encoder (CA-VAE) to better adapt CVAE to the OSR task. Pre-derived attention images are taken as the objective target for reconstruction, thus model is implicitly directed to focus on the class-discriminative regions of the image. In this way, the learned latent representation is de-biased towards class-aware. Experiments on standard image datasets demonstrate the outperformance of the proposed method over existing ones, which achieves new state-of-the-art results. Codes are available at https://github.com/roywang021/CA-VAE.

I. Introduction

In practical classification applications, it is often difficult to collect all possible categories when training classifiers due to various objective factors. A more realistic scenario is Open Set Recognition (OSR) [1], [2], where there is incomplete knowledge during training and the classifiers are required not only to accurately classify the images of known classes but also to detect unknown ones during testing.