PaWI: Parallel Weighted Itemset Mining by Means of MapReduce

Frequent item set mining is an exploratory data mining technique that has fruitfully been exploited to extract recurrent co-occurrences between data items. Since in many application contexts items are enriched with weights denoting their relative importance in the analyzed data, pushing item weights into the item set mining process, i.e., Mining weighted item sets rather than traditional item sets, is an appealing research direction. Although many efficient in-memory weighted item set mining algorithms are available in literature, there is a lack of parallel and distributed solutions which are able to scale towards Big Weighted Data. This paper presents a scalable frequent weighted item set mining algorithm based on the MapReduce paradigm. To demonstrate its action ability and scalability, the proposed algorithm was tested on a real Big dataset collecting approximately 34 millions of reviews of Amazon items. Weights indicate the ratings given by users to the purchased items. The mined item sets represent combinations of items that were frequently bought together with an overall rating above average.

I. Introduction

The rapid evolution of industrial and scientific technologies prompts modern ICT architectures to deal with massive amounts of data. For example, e-commerce platforms, social network systems, and search engines commonly need for storing and processing terabytes of data. The acquired data are large-scale, heterogeneous, and generated at high speed, thus potentially complex to cope with [1]. For example, to plan promotions and advertisements e-commerce companies continuously need to store and analyze millions of electronic transactions made by their customers as well as their degree of satisfaction about their purchased items.