Hadoop and Spark based Rectilinear Range Query Analysis

Many people are curious in what may be learned by querying large amounts of geographical data. Due to the massive size of the data, even with the help of Hadoop/parallel MapReduce's processing, we still need a lot of time to process it, and this prevents us from providing the instantaneous response that is necessary. Most modern database systems, such as Spatial Hadoop or Hadoop-HIS, group data that are near in location together to facilitate indexing and information querying when storing spatial big data. Data querying inside polygonal administrative areas like countries and cities is in high demand. But polygon query is not supported by most database management systems. This work investigates the issue of polygon queries on large geographic datasets and proposes a filtering-refinement approach to speed up calculations. Our technique is shown to greatly minimize compute on the Hadoop or Spatial Hadoop systems, particularly when the polygon is complicated, as shown by our studies utilizing seismic datasets.