Pioneer Learning: Discovering Blind Spots of Transferred Rich Visual Knowledge for Skin Lesion Recognition

Transfer learning has been widely applied in many tasks. However, due to the significant difference in the visual concepts of natural images and medical images, there exists huge gaps between the commonly used ImageNet dataset and the skin lesion datasets. The visual knowledge learned from the natural images may not always robust for modeling subtle lesion patterns from the lesion images or local regions of the lesion images. In this study, we propose a novel learning paradigm, called pioneer learning, to complement and augment the original transfer learning procedure effectively. In particular, we attempt to focus on the learning regarding the subtle lesion patterns. We tend to term these subtle patterns as lesion knowledge blind spots (LKBSs) of the pre-trained models. Specifically, we formulate a series of fast pioneer networks, each of which integrates the pre-trained model with a shallow convolutional neural network (CNN), to efficiently estimate whether a particular lesion image or a local image region belongs to the LKBSs. Subsequently, we delicately design an efficient LKBSs searching strategy to collect LKBSs from all evaluated images and regions carefully. Besides, to alleviate computational costs, we propose a granularity-based region sampling procedure for identifying the LKBSs. Lastly, by mixing the LKBSs with the target skin lesion dataset appropriately, we enable the pre-trained model to focus more on learning new visual concepts from the subtle lesion patterns: LKBSs. In practice, the proposed pioneer learning could improve the convergence speed and the recognition performance of the transferred pre-trained models promisingly. Our method achieves state-of-the-art performance on different skin lesion datasets without using any extra training data.