Research and Implementation of Rule-oriented Subscription Application Based on Stream Computing

With the increasing of video and image data in public security, it is urgent to store massive data and implement relevant application technologies. The public security video and image database system had been effective in real-time storage of massive data and related practical applications. This paper discussed the core processing technology called SparkStreaming, and introduced the core function in public security video and image database system which is rule-oriented subscription application. At the same time, the specific implementation algorithm in the paper was introduced in detail. As a practical case of large data batch computing system, the design of the system framework can build a low latency, high throughput rate, continuous and reliable running streaming system. Through relevant comparative experiments, the data process ability of the algorithm under different operation parameters configuration is obtained. Comparative experiments provided a powerful guidance for the later platform deployment, estimating resources and allocating them reasonably based on the size of data traffic. Other related applications based on SparkStreaming technology can refer to the design and algorithm in this paper, such as rule- oriented disposition application, etc. In the end, this paper proposed the improvement directions in the shortcomings of the algorithm implementation.