An Incremental Principal Component Analysis for Chunk Data

This paper presents a new algorithm of dynamic feature selection by extending the algorithm of Incremental Principal Component Analysis (IPCA), which has been originally proposed by Hall and Martin. In the proposed IPCA, a chunk of training samples can be processed at a time to update the eigenspace of a classification model without keeping all the training samples given so far. Under the assumption that L of training samples are given in a chunk, first we derive a new eigenproblem whose solution gives us a rotation matrix of eigen-axes, then we introduce a new algorithm of augmenting eigen-axes based on the accumulation ratio. We also derive the one-pass incremental update formula for the accumulation ratio. The experiments are carried out to verify if the proposed IPCA works well. Our experimental results demonstrate that it works well independent of the size of data chunk, and that the eigenvectors for major components are obtained without serious approximation errors at the final learning stage. In addition, it is shown that the proposed IPCA can maintain the designated accumulation ratio by augmenting new eigen-axes properly. This property enables a learning system to construct an informative eigenspace with minimum dimensionality.

I. Introduction

Recently, the learning under dynamic environments, where the property of data source is being changed over time, has received a great attention in the computational intelligence community [1]–[4]. In realistic situations, it is not always assumed that a complete set of training samples are given in a batch to learn a system [5], [6]. Therefore, a system is required to test and improve the performance automatically on a on-going basis. This type of learning is often called online learning or incremental learning, and there have been proposed numerous works on this topic so far [7]–[11].