A Self-Optimizing Deep Belief Network with Adaptive-Active Learning

Deep belief network (DBN) is an effective deep learning model, which has been widely used to analyze big-data characteristics, extract features and approximate nonlinear systems. However, due to the deep structure and numerous parameters, DBN suffers from the disadvantage of time-consuming training process generally. To address this problem and improve the training efficiency without reducing accuracy, this paper proposes a self-optimizing deep belief network with adaptive-active learning (SODBN-AAL). In the proposed SODBN-AAL, an adaptive learning algorithm of hyper-parameters is designed to ensure a good accuracy. On this basis, an active learning algorithm is developed based on event-triggered strategy to improve the training efficiency by extracting effective data and skipping invalid data. As a self-optimizing model, SODBN-AAL combines the advantages of both adaptive hyper-parameters and the event-triggered active learning. Two simulation experiments on the benchmark problem and water quality prediction are conducted to show the advantages of SODBN-AAL. The results show that, compared with the basic DBN model, SODBN-AAL averagely improves the learning accuracy by 77.13% and learning efficiency by 84.83%.