Novel Regularization for Learning the Fuzzy Choquet Integral With Limited Training Data

Fuzzy integrals (FIs) are powerful aggregation operators that fuse information from multiple sources. The aggregation is parameterized using a fuzzy measure (FM), which encodes the worths of all subsets of sources. Since the FI is defined with respect to an FM, much consideration must be given to defining the FM. However, in practice this is a difficult task—the number of values in an FM scales as $2^n$, where $n$ is the number of input sources, thus manually specifying an FM quickly becomes tedious. In this article, we review an automatic, data-supported method of learning the FM by minimizing a sum-of-squared error objective function in the context of decision-level fusion of classifiers using the Choquet FI. While this solves the specification problem, we illuminate an issue encountered with many real-world data sets; i.e., if the training data do not contain a significant number of all possible sort orders, many of the FM values are not supported by the data. We propose various regularization strategies to alleviate this issue by pushing the learned FM toward a predefined structure; these regularizers allow the user to encode knowledge of the underlying FM to the learning problem. Furthermore, we propose another regularization strategy that constrains the learned FM's structure to be a linear order statistic. Finally, we perform several experiments using synthetic and real-world data sets and show that our proposed extensions can improve the learned FM behavior and classification accuracy. A previously proposed visualization technique is employed to simultaneously quantitatively illustrate the FM as well as the FI.

I. Introduction

Classification problems generally seek a function, , that can transform or map an observation, , to a prediction or decision, . Machine learning is often utilized to determine a suitable function from a set of training data , where (a vector of labels) and (a set of feature vectors). Learning the prediction function typically involves solving an optimization problem that minimizes the error between the true labels of the training data and the labels predicted by the function .