Comparison of Optimization Techniques for Process Recognition Using Deep Neural Network

This paper brings the topic of comparison of optimization techniques using gradient descent, gradient descent with momentum, Adam and learning rate decay in combination with previous optimization algorithms for face recognition using deep neural network. This paper compares several settings of these techniques as well as techniques among themselves with combination of regularization techniques varying in using no regularization, L2 regularization and dropout to successfully recognize the sex of a person captured. The result of the evaluation is taken in a matter of successfully recognized face rate for train data and test data, cost and deep neural network learning time.