Classifying Incomplete Data with a Mixture of Subspace Experts

Performing inference on data with missing values is a highly common yet challenging task in machine learning, and is often addressed by imputing missing values before performing inference on the filled dataset. However, the two parts of this paradigm may be at odds with one another in that optimal imputation may not yield data that is optimal for inference. In this paper, we propose an ensemble approach to classification with missing values where base learners are trained on embeddings of the data, not on imputed versions of the data. To accomplish this, MoSEs first uses a novel subspace clustering algorithm which characterizes an input space as a hierarchy of localized affine subspaces, each of which produces a low-dimensional embedding of samples with missing data. Then, an expert for each subspace is trained on the associated embedding, with final predictions being a weighted sum of all expert predictions. The degree to which experts are localized to a subspace of data, as well as the degree to which final predictions are influenced by the nearest subspaces are controlled by a pair of tunable hyper-parameters. We perform experiments demonstrating the state-of-the-art performance of this method across a variety of reference datasets.