SRJA: A research on optimizing top-k join queries based on spark

With the explosive growth of RDF data, the traditional centralized environment is increasingly unable to adapt to this huge data query processing requirements, especially the top-k query. This paper designs and implements the original storage scheme of RDF data based on HBase, and presents two top-k query algorithms: STA (Spark Threshold Algorithm) and SRJA (Spark Rank Join Algorithm) which are under Massive RDF Data Query Processing Framework which Based on HBase-based storage system and Spark-based Parallel Computing System. The STA (Spark Threshold Algorithm) is to reduce the RDF data connection during the operation of the operation. The SRJA (Spark Rank Join Algorithm) is proposed to reduce the sort-related operations of the intermediate data and it is the optimization of. Experimental results show that the SRJA algorithm is superior to the STA algorithm and the traditional top-k algorithm, and has strong applicability.