Antiforgetting Incremental Learning Algorithm for Interval Type-2 Fuzzy Neural Network

Sample property drift is an essential issue for interval type-2 fuzzy neural networks (IT2FNNs). When the samples with fresh properties appear, IT2FNN invariably suffers from catastrophic forgetting due to the modification of its numerous parameters. To solve this problem, an antiforgetting incremental learning algorithm is proposed to update IT2FNN. First, a double-displacement indicator (DDI) is designed to detect when catastrophic forgetting occurs caused by property drift. It integrates the indicators from the feature and target spaces to avoid missing detection of property breakpoints. Second, a multilevel learning objective is developed to perceive catastrophic forgetting. The convergence, diversity, and stability criteria of fuzzy rules are embedded into the objective to improve the compatibility of IT2FNN for different properties. Third, an adaptive hierarchical update strategy (AHUS) is proposed to update the parameters of IT2FNN. With AHUS, the parameters are shared among samples with different properties, which can alleviate catastrophic forgetting. Finally, some experiments have verified that the performance of the presented method is superior to other methods in dynamic system identification.

I. Introduction

Interval type-2 fuzzy neural networks (IT2FNNs) with the synergy of fuzzy logic and neural network are widely applied for modeling dynamic systems [1], [2], [3]. In dynamic systems, the statistical properties of the collected samples, including the probability distribution in the feature space and target space change over time, which results in property drift [4], [5]. In a sequential learning scenario, the retrained parameters of IT2FNNs are always prone to overly deviate from the current sample properties. As a result, the historical knowledge acquired by IT2FNN will be covered during the parameters learning. This phenomenon, namely catastrophic forgetting, is usually encountered in dynamic systems modeling of IT2FNNs [6], [7], [8]. To construct an IT2FNN without catastrophic forgetting, it should be clear when forgetting occurs and how to improve the knowledge retention abilities of the network.