Parallelizing Stochastic Gradient Descent with Hardware Transactional Memory for Matrix Factorization

Rapid increase of the amount of available data necessitates large-scale machine learning methods, and Stochastic Gradient Descent (SGD) has become a predominant one of the choices. However, the inherently sequential properties of SGD severely constrain its scalability and prevent it benefiting from multi-core devices. This work parallelizes SGD with transactional memory and leverages hardware support of transactional execution to explore better use of newly deployed features in commercial multi-core processors. To evaluate the performance of our SGD implementation, we compare it with the traditional lock-based approach and conduct quantitative analysis of its synchronization overhead on real world datasets. Experimental results show that the proposed parallelized SGD implementation achieves satisfied scalability and improved execution performance compared with the lock-based approach.