Training Knowledge Inheritance Through Deep Q-Net

When training neural networks, the weights of the model are updated at each optimization step, and the older weights are discarded. In this paper, we propose a method called, Training Knowledge Inheritance (TKI), to use the knowledge about the progression of weight and loss data in reducing overfitting and improving the generalization in the later stages of training. We reformulate the traditional gradient optimization problem as a reinforcement learning task. In particular, by treating the trainable weight space as an environment, the learning rate as action, and the validation accuracies as the rewards, we train a Q-network (controller) to learn the discounted future validation accuracy and guide the later training of another network (worker). We conduct the experiments on the MNIST, CIFAR-10, and CIFAR-100 datasets with naive dense neural networks and ResNet-56. Our results show that TKI could rediscover learning rate schedule rules similar to previous works, including increasing, decaying, and cyclical repeating.

I. Introduction

In recent years, large-scale Deep Learning (DL) models have been demonstrating impressive results in a variety of domains ([1]–[3]). With the popularity of distributed training (e.g. [4], [5]), the cost of training such models has also become larger and larger. On the other hand, at these expensive costs, the data generated during the training process, e.g. weights, performance metrics, loss landscape, etc, obtained through the heavy iterative process are discarded after the training is finished, and often only the final model and weights are saved for inference applications.