Fisher-regularized Support Vector Machine with Pinball Loss Function

Fisher-regularized support vector machine (Fish-erSVM) can approximatively fulfill the Fisher criterion and obtain good statistical separability, which is a combined method of the support vector machine and Fisher discriminant analysis. However, the hinge loss function is related to the shortest distance between two-class sets, and FisherSVM may be hence sensitive to noise. To remedy it, the pinball loss function, which is related to the quantile distance, is introduced into FisherSVM and then a Fisher-regularized support vector machine with pinball loss function (Pin-FisherSVM) is proposed, which combines the noise insensitivity of the pinball loss function with the statistical separability of FisherSVM well. Pin-FisherSVM can be cast as a quadratic programming, which results a globally optimal solution. Experimental results on artificial and real-world datasets demonstrate that our proposed method is insensitive to label noise or feature noise. Compared to FisherSVM, the proposed Pin-FisherSVM has the same computational complexity and exhibits superior noise insensitivity.