Top-k Query Algorithm on Massive Data

In many fields of big data applications, the volume of data has hit a record high and continues to grow exponentially. In decision support applications, one of the challenging issues is to retrieve the most important answers which end-users are interested in from massive data given user-specified preference queries. If users are unsatisfied with the returned results, Why questions may be raised to filter out unwanted objects from the result list. In Top-k queries, it is not easy to specify preferences explicitly. In this context, Why questions are often asked when the unsatisfying query results are retrieved. To deal with the Why questions, a sample-based BTW approach is proposed. By adjusting either the weights of preferences, the k value or both these parameters, this approach can get a new query at a lower cost and thus exclude Why tuples results. However, the BTW approach requires to perform Top-k query processing on each object in the dataset. It results in high time complexity as the number of data increases. To improve the efficiency of the BTW approach, an improved BTW approach is proposed by leveraging two optimization techniques that integrate appropriate pruning strategies. To verify the effectiveness of the solution, a variety of cases are analyzed. Experimental studies using real and synthetic datasets are also conducted to examine the effectiveness and efficiency of the algorithm.