Data Flow Tracking and Real-Time Monitoring Framework of College Online Education Platform based on Spark Big Data Analytic Tools

Data flow tracking and the real-time monitoring framework of college online education platform based on Spark big data analytic tools is studied in the paper. The proposed study designs a highly constrained shared memory model to perform deterministic transformation operations on the resilient and distributed datasets, and these constraints make fault-tolerant implementation less expensive. Spark supports scala, python and java comments and also provides a shell for easy interaction with users. For each node in the Spark cluster, in order to better test the performance of the optimized task scheduling algorithm and with the designed model, the novel real-time monitoring framework of college online education platform is implemented. Through the testing, the overall performance is tested.