Design and Improvement of Recommendation System Based on Spark Framework

With the popularity of mobile Internet, users have a higher demand for real-time personalized recommendation. Traditional recommendation system (RS) often use offline computing to generate recommendation results, which cannot meet the needs of real-time recommendation. Spark Streaming component based on Spark framework can process and analyze real-time data stream, and provide support for real-time recommendation system. The design and improvement of the RS based on Spark framework is designed to meet the requirements of large-scale data processing, real-time recommendation and algorithm improvement and optimization. By utilizing the distributed computing and machine learning capabilities of Spark framework, the performance and effect of the RS can be improved. In this paper, the RS is optimized by means of the Alternating Least Squares (ALS) algorithm through the relevant research on the analysis of important functional requirements of the RS. Based on the key technology of system design and system database design, this paper establishes a RS improvement model and draws the following conclusions: six sets of data samples were selected through simulation experiments. Compared with traditional recommendation algorithms, the improved RS based on Spark framework has improved accuracy and efficiency, and the comprehensive average improvement range is about 7.1 % and 11.2%. This shows that the recommendation improvement system based on Spark framework has a good effect in practical application.