Multitask TSK Fuzzy System Modeling by Mining Intertask Common Hidden Structure

The classical fuzzy system modeling methods implicitly assume data generated from a single task, which is essentially not in accordance with many practical scenarios where data can be acquired from the perspective of multiple tasks. Although one can build an individual fuzzy system model for each task, the result indeed tells us that the individual modeling approach will get poor generalization ability due to ignoring the intertask hidden correlation. In order to circumvent this shortcoming, we consider a general framework for preserving the independent information among different tasks and mining hidden correlation information among all tasks in multitask fuzzy modeling. In this framework, a low-dimensional subspace (structure) is assumed to be shared among all tasks and hence be the hidden correlation information among all tasks. Under this framework, a multitask Takagi-Sugeno-Kang (TSK) fuzzy system model called MTCS-TSK-FS (TSK-FS for multiple tasks with common hidden structure), based on the classical L2-norm TSK fuzzy system, is proposed in this paper. The proposed model can not only take advantage of independent sample information from the original space for each task, but also effectively use the intertask common hidden structure among multiple tasks to enhance the generalization performance of the built fuzzy systems. Experiments on synthetic and real-world datasets demonstrate the applicability and distinctive performance of the proposed multitask fuzzy system model in multitask regression learning scenarios.

I. Introduction

A recent principle tells us that multitask learning or learning multiple related tasks simultaneously has better performance than learning these tasks independently [1]–[5]. Focused on multitask learning, the principal goal of multitask learning is to improve the generalization performance of learners by leveraging the domain-specific information contained in the related tasks [1]. One way to reach the goal is learning multiple-related tasks simultaneously while using a common representation. In fact, the training signals for extra tasks serve as an inductive bias which is helpful to learn multiple complex tasks together [1]. Empirical and theoretical studies on multitask learning have been actively performed in the following three areas: multitask classification learning [2]–[13], multitask clustering [14]–[21], and multitask regression learning [22]–[28]. It has been shown in those studies that, when there are relations between multiple tasks to learn, it is beneficial to learn them simultaneously instead of learning each task independently. Although those studies have indicated the significance of multitask learning and demonstrated certain effectiveness in different real-world applications, the current multitask learning methods are still very limited and cannot keep up with the real-world requirements, particularly in fuzzy modeling. Thus, this paper focuses on multitask fuzzy modeling.