Incremental Feature Learning Using Constructive Neural Networks

Data-driven applications often change over time by considering new features to improve predictive accuracy. Re-training a model from scratch for every change loses the learned knowledge and is very time-consuming. To fill the big literature gap, we devise an incremental feature learning algorithm using constructive neural networks to include new groups of features gradually and without re-learning from scratch. The algorithm dynamically constructs the final model by determining the optimal network topology leading to the best performance. We demonstrate our algorithm’s efficacy through a regression problem by evaluating the sequential models obtained after extending the feature space incrementally, using different feature rankings. We also assess our algorithm without feature grouping and with the non-incremental learning version.