An Implementation of Job Running Backup Function in User-PC Computing System

As a low-cost and high-performance distributed computing platform, we have studied the User-PC Computing (UPC) system based on the master-worker model. Docker container technology is adopted to run various application programs or jobs on heterogeneous PC environments for workers. Some jobs, such as physics simulations and neural networks, require long CPU time, which increases the probability of failure of running workers. The automatic backup of the job running state and migration to other worker will be essential to reduce the job completion delay. In this paper, we implement the job running backup function in the UPC system. Checkpoint-Restore in Userspace (CRIU) is periodically applied to capture the job running state of the running job at a worker. When the master detects the failure, it automatically migrates the job to another worker. To evaluate the function, we conducted experiments using the testbed UPC system with 14 jobs and six workers of different specifications, and confirmed that the proposal successfully resumes the job running from the interrupted point at another worker.