Towards Mitigating Straggler with Deep Reinforcement Learning in Parameter Server

Parameter server paradigm has shown great performance superiority for handling deep learning (DL) applications. One crucial issue in this regard is the presence of stragglers, which significantly retards DL training progress. Previous approaches for solving straggler may not consider the resource utilization of a cluster. This motivates us to make an attempt at designing a new scheme that mitigates straggler problem in DL from the perspective of dynamic balance workloads among workers. To optimize the method of mitigating straggler problem in the traditional parameter server, we propose Help-Reassign Synchronization (HRS) mechanism which has high flexibility to adapt to the dynamic cluster without parameter settings. Furthermore, we propose a Deep Reinforcement Learning (DRL)-based algorithm Decentralized Actor-critic-based Experience Replay (DAER) that can automatically identify and determine helper workers (helper) and helpee workers (helpee). The whole idea has been implemented in a scheme called FlexHS which mitigates straggler problem by creating a dynamic balance between the number of helper and backup overhead. Evaluation under various algorithms evidences the superiority of our scheme.