High-Level Attention Meta-Learner for Class Incremental Learning

Human beings can continuously improve themselves to achieve new goals depending on their environment. Based on this situation, researchers are constantly optimizing machine learning models to enable them to have similar capabilities as humans. As one of the research areas, class incremental learning aims to learn new tasks over time without forgetting. In addition, the aim of meta learning is to integrate the knowledge learned in the past and adapt to the current task more quickly. Based on the above, we use meta learning to alleviate the catastrophic forgetting problem in class incremental learning. We propose a method, named High-Level Attention Meta-Learner (HLAML), which learns shared parameters of all tasks and predicts the task automatically. HLAML contains an attention module at the high-level of the network which can extract task information, thus helping the model to better distinguish past tasks. Furthermore, for the attention module, we design a task loss function to help the attention module better extract task-related features. Our model achieves excellent performances on MNIST, SVHN, CIFAR10, CIFAR100, ImageNet100, and ImageNet1000.