A Fast Non-Negative Latent Factor Model Based on Generalized Momentum Method

Non-negative latent factor (NLF) models can efficiently acquire useful knowledge from high-dimensional and sparse (HiDS) matrices filled with non-negative data. Single latent factor-dependent, non-negative and multiplicative update (SLF-NMU) is an efficient algorithm for building an NLF model on an HiDS matrix, yet it suffers slow convergence. A momentum method is frequently adopted to accelerate a learning algorithm, but it is incompatible with those implicitly adopting gradients like SLF-NMU. To build a fast NLF (FNLF) model, we propose a generalized momentum method compatible with SLF-NMU. With it, we further propose a single latent factor-dependent non-negative, multiplicative and momentum-incorporated update algorithm, thereby achieving an FNLF model. Empirical studies on six HiDS matrices from industrial application indicate that an FNLF model outperforms an NLF model in terms of both convergence rate and prediction accuracy for missing data. Hence, compared with an NLF model, an FNLF model is more practical in industrial applications.

I. Introduction

A big data-related application commonly involves a huge amount of entities, e.g., millions of users are involved in a social network service system [1]–[3]. In such an application, interactions among numerous entities result in high-dimensional relationships. Moreover, since each entity cannot establish complete interactions with the others, e.g., a user cannot touch all items in a recommender system [4]–[6], such high dimensional relationships can be very sparse. High-dimensional and sparse (HiDS) matrices, where numerous entries are unknown rather than zeroes, are frequently adopted to describe such relationships [1]–[10].