Autonomous Choice of Deep Neural Network Parameters by a Modified Generative Adversarial Network

The choice of parameters, and the design of the network architecture are important factors affecting the performance of deep neural networks. However, this task still heavily depends on trial and error, and empirical results. Considering that there are many design and parameter choices, it is very hard to cover every configuration, and find the optimal structure. In this paper, we propose a novel method that autonomously and simultaneously optimizes multiple parameters of any given deep neural network by using a modified generative adversarial network (GAN). In our approach, two different models compete and improve each other progressively. Without loss of generality, the proposed method has been tested with three different neural network architectures, and three very different datasets and applications. The results show that the presented approach can simultaneously and successfully optimize multiple neural network parameters, and achieve increased accuracy in all three scenarios.

1. INTRODUCTION

Training of deep learning methods requires large amounts of data, and they usually perform better when training data size is increased. Yet, for some applications, it is not always possible to obtain more data when the existing dataset is not large enough. Even though the raw data can be collected easily, data labeling or annotation is difficult, expensive and time consuming. Successors of [1] yielded better accuracies with less number of parameters on the same benchmark with some architectural modifications using the same building blocks. This shows that the choice of parameters, and the design of the architecture are important factors affecting the performance. Many researchers proposed different CNN architectures [2], [3], [4], [5], [6], [7], [8] to achieve higher accuracy.