Mutual Learning for Pattern Recognition

Mutual learning algorithm can be an efficient mechanism for improving the machine learning and neural network efficiency in a multi-agent system. Specifically, in many cases, where the system cannot be trained using a big training dataset, the data exchange in teacher-student network system can lead to efficient learning. Usually, in mutual learning algorithms, a big network plays the role of a static teacher and passes the data to smaller networks, known as student networks, to improve the efficiency of the latter. In this paper, we will show that two small networks can dynamically play the changing roles of teacher and student to share their knowledge and hence, the efficiency of both the networks improve simultaneously. We demonstrate the concept and the proposed mutual learning algorithm using convolutional neural networks (CNNs) to recognize the benchmark Modified National Institute of Standards and Technology (MNIST) hand-writing dataset.