Transfer Learning Algorithm for Image Classification Task and its Convergence Analysis

Theoretical analysis of transfer learning of the deep neural networks (DNN) is crucial in ensuring stability or convergence and gaining a better understanding of the networks for further development. However, most current transfer learning methods are black-box approaches that are more focused on empirical studies. This paper develops a transfer learning algorithm for deep convolutional neural networks (CNN) with batch normalization layers. A convergence-guaranteed transfer learning algorithm is proposed to train the classifier of a deep CNN with pretrained convolutional layers. Two classification case studies based on VGG11 with the MNIST dataset and CIFAR10 dataset are presented to demonstrate the performance of the proposed approach and explore the effect of batch normalization layers on transfer learning,