Training data reduction and nonlinear feature extraction in classification based on greedy Generalized Discriminant Analysis

Generalized Discriminant Analysis (GDA) shows a powerful nonlinear feature extraction technique by kernel tricks. The size of its kernel matrix increases quadratically with the number of training data. For large training data set, it suffers from computational problem of diagonal and occupies large storage space of kernel matrix. Here, a more efficient nonlinear feature extraction method, Greedy Generalized Discriminant Analysis (GGDA) is presented to training data reduction and nonlinear feature extraction in classification. The simulation results indicate that the GGDA method reduces computational complexity due to the reduced training set in classification while retaining the performance of the GDA method.

I. Introduction

Generalized Discriminant Analysis (GDA) method in [1] is the nonlinear extension of the ordinary Linear Discriminant Analysis (LDA) method by kernel tricks in [2]. It shows a powerful nonlinear feature extraction technique. However, the training time and evaluation costs of the GDA method are dependant on the size of the training data. During training, the kernel matrix, which grows quadratically with the number of samples in the training data in [3], needs to be calculated before the LDA method can be applied in feature space. For large scale data set, it suffers from computational problem of diagonal and occupies large storage space of kernel matrix. The size of the training data is therefore vital in any real system incorporating the GDA method. So, a more efficient nonlinear feature extraction method, Greedy Generalized Discriminant Analysis (GGDA) method in [4] is presented to reduce training data and nonlinear feature extraction in classification.