Contents I. Introduction
Researchers have successfully applied fuzzy neural networks (FNNs) in several areas, such as classification and regression. The fuzzy if–then rule derivation is often difficult and time consuming, and it requires expert knowledge. FNNs provide a solution to address the common bottleneck in fuzzy system design. Many researchers have proposed FNNs with parameter and/or structure learning [1]–[15]. In [1] an adaptive-network-based fuzzy inference system (ANFIS) is proposed. The ANFIS is learned through parameter learning. The structure of the ANFIS is fixed, and the input space is partitioned in a grid type. This kind of partition faces the curse of dimensionality because the number of fuzzy rules increases exponentially as the dimension of the input space increases. To address this problem, many FNNs with offline [3] or online [2] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] structure-learning ability have been proposed. These structure-learning algorithms determine the number of rules using the idea of clustering. The objective is to design a well-performed FNN with the least number of rules. However, the problems considered in the FNNs [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [15] are regression problems, where the network input dimensions are smaller than 10. For these low-dimensional regression problems, an FNN with a small rule set may achieve satisfactory performance. For many problems, such as pattern classification, tens or hundreds of attributes are fed as inputs to an FNN. These high-dimensional classification problems usually require an FNN with a large rule set to achieve good performance. Even an FNN with a small rule set may achieve good performance for some of these high-dimensional problems, and the user may need to try a larger rule set to see whether or not much better performance can be achieved. To train an FNN with a large rule set for high-dimensional problems is a computationally intensive task. The aforementioned FNNs are implemented on a single-threaded CPU, where datasets are processed in series instead of in parallel. For some problems, FNN training on a CPU may take several days. FNNs are suitable for implementation on parallel processing units because they can be expressed as data-parallel computations due to the parallel processing property of fuzzy rules and input variables. This property motivates the proposal to implement FNN using graphic processing units (GPUs) [16]. The GPU-implemented FNN (GPU-FNN) that are considered in this paper uses the structure and parameter learning in a self-constructing neural fuzzy inference network (SONFIN) [2]. Like the FNNs in studies [3]–[12] and [15], the structure and parameter learning algorithms in the SONFIN were proposed to design a well-performed fuzzy system with the least number of rules. This study uses SONFIN learning because of its powerful learning ability with a simple structure-learning approach. As mentioned earlier, for the problems with high-dimensional inputs, a large number of rules may be inevitable to achieve good performance even with network structure/parameter learning. The GPU-FNN is proposed to address this computationally intensive problem.
Structure of the GPU-FNN.
