On rival penalization controlled competitive learning for clustering with automatic cluster number selection | IEEE Journals & Magazine | IEEE Xplore

On rival penalization controlled competitive learning for clustering with automatic cluster number selection


Abstract:

The existing rival penalized competitive learning (RPCL) algorithm and its variants have provided an attractive way to perform data clustering without knowing the exact n...Show More

Abstract:

The existing rival penalized competitive learning (RPCL) algorithm and its variants have provided an attractive way to perform data clustering without knowing the exact number of clusters. However, their performance is sensitive to the preselection of the rival delearning rate. In this paper, we further investigate the RPCL and present a mechanism to control the strength of rival penalization dynamically. Consequently, we propose the rival penalization controlled competitive learning (RPCCL) algorithm and its stochastic version. In each of these algorithms, the selection of the delearning rate is circumvented using a novel technique. We compare the performance of RPCCL to RPCL in Gaussian mixture clustering and color image segmentation, respectively. The experiments have produced the promising results.
Published in: IEEE Transactions on Knowledge and Data Engineering ( Volume: 17, Issue: 11, November 2005)
Page(s): 1583 - 1588
Date of Publication: 30 November 2005

ISSN Information:

References is not available for this document.

1 Introduction

Competitive learning has been widely applied to a variety of applications such as vector quantization [9], [14], data visualization [8], [13], and particularly to unsupervised clustering [1], [6], [21], [24]. In the literature, k-means [15] is a popular competitive learning algorithm, which trains seed points (also called units hereinafter), denoted as , in a way that they converge to the data cluster centers by minimizing the mean-square-error (MSE) function. In general, k-means algorithm has at least two major drawbacks: 1) It suffers from the dead-unit problem. If the initial positions of some units are far away from the inputs (also called data points interchangeably) in Euclidean space compared to the other units, these distant units will have no opportunity to be trained and, therefore, immediately become dead units. 2) If the number of clusters is misspecified, i.e., is not equal to the true cluster number , the performance of k-means algorithm deteriorates rapidly. Eventually, some of the seed points are not located at the centers of the corresponding clusters. Instead, they are either at some boundary points between different clusters or at points biased from some cluster centers [24].

Select All
1.
S.C. Ahalt, A.K. Krishnamurty, P. Chen and D.E. Melton, "Competitive Learning Algorithms for Vector Quantization", Neural Networks, vol. 3, pp. 277-291, 1990.
2.
H. Akaike, "Information Theory and an Extension of the Maximum Likelihood Principle", Proc. Second Int’l Symp. Information Theory, pp. 267-281, 1973.
3.
H. Akaike, "A New Look at the Statistical Model Identfication", IEEE Trans. Automatic Control, pp. 716-723, 1974.
4.
H. Bozdogan, "Model Selection and Akaike’s Information Criterion: The General Theory and Its Analytical Extensions", Psychometrika, vol. 52, no. 3, pp. 345-370, 1987.
5.
J. Breckenridge, "Replicating Clustering Analysis: Method Consistency and Validity", Multivariate Behavioural Research, 1989.
6.
E.W. Forgy, "Cluster Analysis of Multivariate Data: Efficiency Versus Interpretability of Classifications", Proc. Biometric Soc. Meetings, 1965.
7.
J. Fridlyand and S. Dudoit, "Applications of Resampling Methods to Estimate the Number of Clusters and to Improve the Accuracy of a Clustering Method", Sept. 2001.
8.
B. Fritzke, "Growing Cell Structures—A Self-Organizing Network for Unsupervised and Supervised Learning", Neural Networks, vol. 7, no. 9, pp. 1441-1460, 1994.
9.
R.M. Gray, "Vector Quantization", IEEE ASSP Magazine, vol. 1, pp. 4-29, 1984.
10.
P. Guo, C.L. Philip Chen and M.R. Lye, "Cluster Number Selection for a Small Set of Samples Using the Bayesian Ying-Yang Model", IEEE Trans. Neural Networks, vol. 13, no. 3, pp. 757-763, 2002.
11.
M. Har-even and V.L. Brailovsky, "Probabilistic Validation Approach for Clustering", Pattern Recognition Letters, vol. 16, pp. 1189-1196, 1995.
12.
T. Lange, M.L. Braun, V. Roth and J.M. Buhmann, "Stability-based Model Selection", Proc. Neural Information Processing Systems, 2002.
13.
T. Kohonen, "Self-Organized Formation of Topologically Correct Feature Maps", Biological Cybernetics, vol. 43, pp. 59-69, 1982.
14.
Y. Linde, A. Buzo and R.M. Gray, "An Algorithm for Vector Quantizer Design", IEEE Trans. Comm., pp. 84-95, 1980.
15.
J.B. MacQueen, "Some Methods for Classification and Analysis of Multivariate Observations", Proc. Fifth Berkeley Symp. Math. Statistics and Probability, vol. 1, pp. 281-297, 1967.
16.
T.M. Martinetz and K.J. Schulten, "A ’Neural-Gas’ Network Learns Topologies", Artificial Neural Networks, pp. 397-402, 1991.
17.
V. Olman, D. Xu and Y. Xu, "Cubic: Identification of Regulatory Binding Sites Through Data Clustering", J. Bioinformatics and Computational Biology, vol. 1, no. 1, pp. 21-40, 2003.
18.
G. Schwarz, "Estimating the Dimension of a Model", The Annals of Statistics, vol. 6, no. 2, pp. 461-464, 1978.
19.
R. Tibshirani, G. Walther and T. Hastie, "Estimating the Number of Clusters via the Gap Statistic", J. Royal Statistical Soc. Series B, vol. 63, pp. 411-423, 2001.
20.
R. Tibshirani, G. Walther, D. Botstein and P. Brown, "Cluster Validation by Prediction Strength", Sept. 2001.
21.
H.Q. Wang and D.S. Huang, "A Novel Clustering Analysis Based on PCA and SOMs for Gene Expression Patterns" in Lecture Notes in Computer Science, Springer-Verlag, vol. 3174, pp. 476-481, 2004.
22.
L. Xu, "How Many Clusters?: A Ying-Yang Machine Based Theory for a Classical Open Problem in Pattern Recognition", Proc. IEEE Int’l Conf. Neural Networks, vol. 3, pp. 1546-1551, 1996.
23.
L. Xu, "Bayesian Ying-Yang Machine Clustering and Number of Clusters", Pattern Recognition Letters, vol. 18, no. 11-13, pp. 1167-1178, 1997.
24.
L. Xu, A. Krzyżak and E. Oja, "Rival Penalized Competitive Learning for Clustering Analysis RBF Net and Curve Detection", IEEE Trans. Neural Networks, vol. 4, pp. 636-648, 1993.
Contact IEEE to Subscribe

References

References is not available for this document.