Application Research for real time computing

In the era of big data and artificial intelligence, the key performance indicators of real-time computing encounter many challenges. It is very difficult to realize real-time computing by using traditional computing tools. This research will introduce the specific application research of Flink in real-time computing through the production and practice process of Apache Flink in real-time computing. Through the research, it is found that Flink has good performance in massive data, real-time ETL, real-time risk control, distributed call chain analysis and other application cases. Research and propose specific Flink platform construction technology methods, and put forward improvement measures for several Flink problems encountered in the process of Flink platform.

I. Evolution and Challenges in Real-Time Computing

Real time calculation is an online calculation based on some disordered time series data that arrive in real time, the rate is uncontrollable, the arrival order is independent and does not guarantee the order, and cannot be replayed unless specially saved. Therefore, in real-time computing, there will be problems such as data disorder, data delay, and inconsistency between event time and processing time. The number of peak events of the platform reached 11 million / s, and it encountered considerable challenges in correctness, fault tolerance, performance, delay, throughput, scalability, etc.