Proactive, Accuracy-aware Straggler Mitigation in Machine Learning Clusters

Slower workers, known as stragglers, can signifi-cantly prolong training time in Machine Learning (ML) clusters. We present SMS, a proactive straggler mitigation system with four key components. Firstly, instead of reacting after stragglers occur, SMS predicts potential stragglers using ML and manages them preemptively. Secondly, for predicted stragglers, we reduce their batch sizes while preserving accuracy by keeping only one copy of similar data samples in a batch. Thirdly, to tackle stragglers due to network congestion in the Parameter Server (PS) architecture, we avoid transferring parameters similar to those from the previous iteration from predicted stragglers. Fourthly, in the all-reduce architecture, we update the communication sequence among workers to minimize gradient transfer time in an iteration. Our experimental results show that SMS reduces up to 53% training time while preserving the training accuracy when compared with existing methods.