RAFNet: Interdomain Representation Alignment and Fine-Tuning for Image Series Classification

Classification of remote sensing image series, which differs in quality and details, has important implications for the analysis of land cover, whereas it is expensive and time-consuming as a result of manual annotations. Fortunately, domain adaptation (DA) provides an outstanding solution to the problem. However, information loss while aligning two distributions often exists in traditional DA methods, which impacts the effect of classification with DA. To alleviate this issue, an inter-domain representation alignment and fine-tuning-based network (RAFNet) is proposed for image series classification. Interdomain representation alignment, which is fulfilled by a variational autoencoder (VAE) trained by both source and target data, encourages reducing the discrepancy between the two marginal distributions of different domains and simultaneously preserving more data properties. As a result, RAFNet, which fuses the multiscale aligned representations, performs classification tasks in the target domain after being well-trained with supervised learning in the source domain. Specifically, the multiscale aligned representations of RAFNet are acquired by duplicating the frozen encoder of VAE. Then, an information-based loss function is designed to fine-tune RAFNet, in which both the unchanged information and the changed information implied in change maps are completely used to learn the discriminative features better and make the model more generalized for the target domain. Finally, experiment studies on three datasets validate the effectiveness of RAFNet with considerable segmentation accuracy even though the target data have no access to any annotated information.