Optimizing hyperparameters of linear SVM classifiers using the support method

The support vector machine (SVM) is a popular classifier known for its powerful generalization capabilities and its wide application in a variety of fields. In order to achieve optimal performance of this model, the user is required to provide values for the tuning parameters: the regularization cost parameter and kernel parameters. In this work, we focus on the optimization of the regularization parameter, by developing a new algorithm based on the direct support method that allows for computing all breakpoints that minimize the number of misclassified examples and maximize the margin. This approach is iterative and formed by two phases. Phase 1 is used to calculate the first optimal solution corresponding to the initial value of the regularization parameter
$A$
= 1/C, while phase 2 is used to iteratively calculate the rest of the breakpoints.