A Multilayered-and-Randomized Latent Factor Model for High-Dimensional and Sparse Matrices

How to extract useful knowledge from a high-dimensional and sparse (HiDS) matrix efficiently is critical for many big data-related applications. A latent factor (LF) model has been widely adopted to address this problem. It commonly relies on an iterative learning algorithm like stochastic gradient descent. However, an algorithm of this kind commonly consumes many iterations to converge, resulting in considerable time cost on large-scale datasets. How to accelerate an LF model's training process without accuracy loss becomes a vital issue. To address it, this study innovatively proposes a multilayered-and-randomized latent factor (MLF) model. Its main idea is two-fold: a) adopting randomized-learning to train LFs for implementing a ‘one-iteration’ training process for saving time; and 2) adopting the principle of a generally multilayered structure as in a deep forest or multilayered extreme learning machine to structure its LFs, thereby enhancing its representative learning ability. Empirical studies on six HiDS matrices from real applications demonstrate that compared with state-of-the-art LF models, an MLF model achieves significantly higher computational efficiency with satisfactory prediction accuracy. It has the potential to handle LF analysis on a large scale HiDS matrix with real-time requirements.

1 Introduction

The RAPID expanding World-Wide-Web brings human society into an era of big data. In big data–related applications, interaction data among numerous entities commonly provide rich information reflecting their behaviors, e.g., user experience and service performance in quality-of-service data from service-oriented applications [1], [2], [3], node connections and characteristics in node interaction data from wireless sensor networks [4], [5], [6], and user preferences and item popularity in user-item ratings data from recommender systems [7], [8], [9], [10]. Moreover, due to the exploding numbers of involved entities, it is impossible to observe the full relationship among them [1], [2], [5], [6], [7].