Efficient Data Loading for Deep Neural Network Training

Deep neural networks (DNNs) have achieved outstanding results in a wide range of applications. However, A DNN training is a data- and compute-intensive task to obtain high accuracy. Multiple state-of-the-art AI accelerators such as GPUs can be deployed in a single machine for training, data loading dominates a significant amount of time in the whole training process. As a result, data loading becomes a key bottleneck to limit the whole training performance. In this paper, we present efficient data loading for DNN trainings by using two methods: Based on data access pattern and efficient utilization of disk I/O, we use a block as the unit for data reading. As data are repeated used in training an accurate model, we cache partially loaded data for faster access. We use ImageNet as the dataset and present comprehensive evaluations in terms of running time and accuracy for state-of-the-art deep neural networks on a machine with four V100 GPU cards. The evaluation results demonstrated that while the accuracy is maintained, our optimizations can achieve up to 10 fold performance growth in terms of the whole run time for measured DNN trainings.