EXPANSE, A Continual Deep Learning System; Research Proposal

The final goal of the Artificial Intelligence community is to achieve Artificial General Intelligence, a human level of intelligence with precision, persistence, and processing speed of computers. As humans, we have the ability to progress via continual learning. We always add a skill on top of our past learned skills. Deep transfer learning, followed by progressive learning, is attempting to mimic human skill learning ability, a task that is endless for human beings. However, even the most advanced deep learning models need to be trained on a massive dataset and still fail to face the same task in a different context. Progressive Learning, a subcalegory of deep transfer learning, is the closest technique to human continual learning ability. The main goal of the following proposed system is to drive the current progressive learning method a step closer to the final destination of Artificial General Intelligence. The current progressive learning method is focused on adding a fresh layer(s) to the end of a pre-trained model for a slightly different task or dataset. The idea behind EXPANSE is that we can tackle a more distant task and/or dataset by expanding all layers of the model, also adding a fresh layer(s) al the end In other words, we expand the network vertically and horizontally. The whole, old part of the network will be used frozen, and only the freshly added part should be trained on target data. Moreover, EXPANSE is introducing two steps of training inspired by the human educational system.