Guiding Hidden Layer Representations for Improved Rule Extraction From Neural Networks

The production of relatively large and opaque weight matrices by error backpropagation learning has inspired substantial research on how to extract symbolic human-readable rules from trained networks. While considerable progress has been made, the results at present are still relatively limited, in part due to the large numbers of symbolic rules that can be generated. Most past work to address this issue has focused on progressively more powerful methods for rule extraction (RE) that try to minimize the number of weights and/or improve rule expressiveness. In contrast, here we take a different approach in which we modify the error backpropagation training process so that it learns a different hidden layer representation of input patterns than would normally occur. Using five publicly available datasets, we show via computational experiments that the modified learning method helps to extract fewer rules without increasing individual rule complexity and without decreasing classification accuracy. We conclude that modifying error backpropagation so that it more effectively separates learned pattern encodings in the hidden layer is an effective way to improve contemporary RE methods.

I. Introduction

Error backpropagation is the most widely used supervised learning method for neural networks and has achieved success in many classification and prediction applications. A typical network has an architecture consisting of an input layer, one or more hidden layers, and an output layer (Fig. 1). There are several variants of error backpropagation, usually driven by minimizing the sum of squared error between the actual output values and the target teaching signals. The network learns a mapping between the input and output units, while the hidden units and the weights between them and other units contain the network's internal representation of the input. This distributed representation as large matrices of floating point numbers makes it very difficult for a person to understand what a trained network has learned. This difficulty has inspired substantial past research on how to extract symbolic human-readable rules from a network so that one can be more confident about its classifications and understand more about what has been learned from the data. In spite of a large amount of work addressing this issue ([1]–[5]), the results obtained are still very limited. Fig. 1.

Typical fully connected feedforward neural network.