Efficient Regression with Feature Selection Based on L12-Norm

Feature selection is one of the most important components of many machine learning applications. Most of the existing sparse coding-based methods only concerned with the regularization of the transformation matrix, but ignoring the sparsity of projected features in the intrinsic feature space. In this paper, we introduce a novel regularization with ℓ
12
-norm, called class specificity distribution term. It aims to ensure that samples with the same label could obey a specific distribution, while data from the different categories should have different distributions in the intrinsic feature space. Then we propose a novel embedded feature selection framework based on regression and class specificity distribution term measured by ℓ
12
-norm, termed RL12. An efficient augmented Lagrange multipliers based optimization algorithm is proposed to solve the ℓ
12
-norm regularization iteratively. Experimental results demonstrate the effectiveness of our method.