TSK Fuzzy System Optimization for High-Dimensional Regression Problems

To simultaneously resolve the curse of dimensionality and preserve the strong interpretability of Takagi-Sugeno-Kang (TSK) fuzzy systems, linear dimensionality reduction with simple geometric interpretation is necessary. Existing approaches usually perform dimensionality reduction and TSK fuzzy system training separately, leading to suboptimal performance. This paper proposes consistent dimensional reduction (CDR), which automatically identifies a shared informative input subspace for the rule antecedents and consequents via gradient backpropagation, and thus performs well-generalizable end-to-end consistent feature extraction. Integrating CDR with our previously proposed MBGD-RDpA (mini-batch gradient descent with regularization, DropRule, and Powerball AdaBelief) leads to CDR-GRDpA for grid partition based TSK fuzzy system optimization, and CDR-FRDpA for fuzzy c-means clustering based TSK fuzzy system optimization. Experiments demonstrated that CDR-FRDpA almost always obtained the smallest root mean squared error, with high computational efficiency and strong parameter robustness. Further integrating CDR-FRDpA with adaptive rule pruning or adaptive group LASSO leads to two variants with better generalization, albeit slightly increased computational cost.