Design of a self-organizing modular neural network with inter-module connections

To further promote the generalization ability of modular neural networks (MNNs) and design a network that approximates the modular structure of the brain, a self-organizing MNN with inter-module connections (SOIMNN) is designed in this study. Firstly, a soft clustering method is employed to divide the feature dimensions of datasets to generate several feature subsets. Secondly, subnetworks with the same number of subsets are used to learn subtasks. Each sub network adopts a feedforward neural network with double hidden layers, and connections between different subnetworks are constructed. At the same time, a gradient-based growth mechanism and a threshold-based pruning strategy are employed to adjust the structure of the SOIMNN, and all parameters are learned by the improved Levenberg-Marquart algorithm. Finally, the output of SOIMNN is obtained from the weighted average of each subnetwork. Experimental results on several benchmark datasets demonstrate that SOIMNN has better generalization performance and more compact structure than other MNNs.