Fisher Discriminant Analysis Random Forest for Online Class Incremental Learning

Class incremental learning needs to deal with a dynamic environment where data class appears incrementally, it is a challenge to learn new knowledge while preserving what has already been learned. On the other hand, due to the limited storage of the online scenario, algorithm is usually obstructed to frequently scan or simply store all historical data, it is another challenge to reduce the historical data storage for algorithm. Few existing work have addressed above challenges simultaneously. In this paper, we propose Fisher Discriminant Analysis Random Forest (FDARF), which consists of two parts, GHS (Generate Hierarchical Split) and RRS (Random Reform Subtree), that cooperatively operate. GHS combines FDA (Fisher Discriminant Analysis) with tree hierarchy to learn a hierarchical split of data space that provides strong ability for classification. The statistics in leaves (i.e. historical data) can be described by covariance matrix and further optimized by matrix sketching algorithm to reduce storage; for every tree initialized by GHS, RRS randomly reforms certain state subtree, which creates diversity that can be ensemble for ensuring effectiveness of class incremental learning. Extensive experiments on diverse datasets validate that FDARF can well adapt to the online class incremental learning.