A Deep Learning Framework for Automated Transfer Learning of Neural Networks

Transfer Learning is a technique that reduces the time taken for training and improves performance by reusing the weights of a previously trained source neural network. This poses a question of how the source network must be chosen, which is still an unsolved problem. In this work, we have built a framework that automatically performs transfer learning by selecting the source neural network based on an estimate of dataset classification difficulty. The framework designates the neural network of the dataset that is closest in difficulty as the source. The framework is evaluated on 7 datasets namely SVHN, Cifar10, Cifar100, GTSRB, MNIST, Flowers and Linnaeus5, and experimental results suggest that in most cases this type of source selection gives the highest improvement in accuracy. The framework provides an average improvement in accuracy of 6.7% for ResNet than when training from scratch, and achieves it within the first 10 epochs in some cases.