![Data Clustering Relevant Clustering Algorithms Clustering validation
Applications of Clustering:
The applications of clustering are[2]:
1.) Its an intermediate step for other fundamental data mining
problems.
2.) For Collaborative filtering.
3.) Customer Segmentation.
4.) Data summarisation.
5.) Multimedia data analysis.
6.) Biological data analysis.
7.) Social Network Analysis.
etc.](https://image.slidesharecdn.com/dataclusteringinternshippresentation-160629114759/85/Data-clustering-4-320.jpg)
![Data Clustering Relevant Clustering Algorithms Clustering validation
K-means algorithm
An unsupervised learning algorithm.
Applies on the m-dimensional hyperspace,for a given data set.
The pre-processing steps are: ’Handling missing values’ and
’Scaling’.
Scaling :
If the attribute is A.and have range [Amin, Amax ].Then, to scale a
value of A as A x, the formula is:
Ax (scaled) = (Ax - Amin)/(Amax - Amin)](https://image.slidesharecdn.com/dataclusteringinternshippresentation-160629114759/85/Data-clustering-10-320.jpg)
![Data Clustering Relevant Clustering Algorithms Clustering validation
Fuzzy C-means Algorithm
And fill the value of wij in matrix W.
Step 3.) For each cluster,do :
3.i) Calculate the new position(or value) of centroid,ck by :
3.ii) Shift the centroid to the calculated position(or value) in
previous step.
Step 4.) 4.i) compute and update values of elements of W (k) as
W (k+1
4.ii) If ||W (k+1) − W (k)|| > β, then go to Step 2. Else The present
value of W is the resultant partitioning matrix.
where, k is the iteration step.
β is the termination criterion, β ∈ [0, 1].
’W (k)’ is the fuzzy membership matrix in kth iteration.](https://image.slidesharecdn.com/dataclusteringinternshippresentation-160629114759/85/Data-clustering-21-320.jpg)
![Data Clustering Relevant Clustering Algorithms Clustering validation
Advantages and disadvantages of K-means and Fuzzy
C-means Algorithms:
The K-means algorithm is fast, robust and easier to
understand. FCM is more complex that K-means.
K-means gives better result when data set are distinct or well
separated from each other, that is for non-overlapping
clusters.FCM is better for overlapping clusters.
The limitations of both is that, the value of K must be known
priorly,
K-mean results in local optima,it is applicable only when
mean is defined, hence fails for categorical data and is unable
to handle noisy data and outliers[1]
The time complexity of the K-Means algorithm is O(tkdn) and
the time complexity of FCM algorithm is O(ndk2t) where, n
is number of data objects, k is number clusters, d is dimension
of each object and t is iterations. Normally, k, t, d << n[5]](https://image.slidesharecdn.com/dataclusteringinternshippresentation-160629114759/85/Data-clustering-23-320.jpg)
![Data Clustering Relevant Clustering Algorithms Clustering validation
Cluster validation
Evaluation of clustering results sometimes is referred to as cluster
validation.[7]
Measures are used to compare the quality of different clustering
algorithms.
The measures are classified as:
Internal Indices.
External Indices.](https://image.slidesharecdn.com/dataclusteringinternshippresentation-160629114759/85/Data-clustering-26-320.jpg)
![Data Clustering Relevant Clustering Algorithms Clustering validation
Dunn and Dunn index:
It is proposed by [3].
Identifies clusters which are well separated and compact.
Goal is[6]:
to maximize the inter-cluster distance.
minimizing the intra-cluster distance.
The Dunn index for k clusters is defined by:](https://image.slidesharecdn.com/dataclusteringinternshippresentation-160629114759/85/Data-clustering-29-320.jpg)
![Data Clustering Relevant Clustering Algorithms Clustering validation
Davies Bouldin index:
The Davies Bouldin index [8] is based on similarity measure of
clusters (Rij ).
Dispersion(si ) of a cluster and dissimilarity between
clusters(dij ) are used to compute the Davies-Bouldin(DB)
index.
Similarity measure of clusters, (Rij ) must satisfy the
conditions:
Rij >= 0
Rij = Rji
if si = 0 and sj = 0 then Rij = 0
if sj >sk and dij = dik then Rij >Rik
if sj = sk and dij <dik then Rij >Rik](https://image.slidesharecdn.com/dataclusteringinternshippresentation-160629114759/85/Data-clustering-33-320.jpg)
![Data Clustering Relevant Clustering Algorithms Clustering validation
Davies Bouldin index:
Usually Rij is defined in the following way[4]:
Then the Davies Bouldin index is defined as:
where,](https://image.slidesharecdn.com/dataclusteringinternshippresentation-160629114759/85/Data-clustering-34-320.jpg)
![Appendix
References I
[1] site/dataclusteringalgorithms/k-means-clustering-algorithm.
[2] Aggrawal, C. C., and Reddy, C. K.
Data Clustering.
CRC Press.
[3] Dunn, J.
Well separated clusters and optimal fuzzy partitions.
Journal of Cybernetics 4 (1974), 95104.
[4] Ferenc Kovcs, Csaba Legny, A. B.
Cluster validity measurement techniques.](https://image.slidesharecdn.com/dataclusteringinternshippresentation-160629114759/85/Data-clustering-36-320.jpg)
![Appendix
References II
[5] Ghosh, S., and Dubey, S. K.
Comparative analysis of k-means and fuzzy c-means
algorithms.
((IJACSA) International Journal of Advanced Computer
Science and Applications 4(4) (2013).
[6] Sandro Saitta, B. R., and Smith, I. F.
A bounded index for cluster validity.
[7] wikipedia.
https://en.wikipedia.org/wiki/cluster analysis#evaluation and assessm](https://image.slidesharecdn.com/dataclusteringinternshippresentation-160629114759/85/Data-clustering-37-320.jpg)
Data clustering
- 1. Data Clustering Relevant Clustering Algorithms Clustering validation Data Clustering An Unsupervised Learning Approach Garima Shakya [email protected] Department of Computer Science and Technology IIEST,Shibpur,Howrah 28 June 2016
- 2. Data Clustering Relevant Clustering Algorithms Clustering validation Outline 1 Data Clustering Feature Selection Methods: Distance based Algorithm 2 Relevant Clustering Algorithms K-means algorithm Fuzzy C-means Algorithm Advantages and disadvantages of K-means and Fuzzy C-means Algorithms 3 Clustering validation Dunn and Dunn index Davies Bouldin index
- 3. Data Clustering Relevant Clustering Algorithms Clustering validation Data Clustering: ”The task of grouping a set of objects in such a way that objects in same group (called a cluster) are more similar to each other than to those in other groups(clusters)”.
- 4. Data Clustering Relevant Clustering Algorithms Clustering validation Applications of Clustering: The applications of clustering are[2]: 1.) Its an intermediate step for other fundamental data mining problems. 2.) For Collaborative filtering. 3.) Customer Segmentation. 4.) Data summarisation. 5.) Multimedia data analysis. 6.) Biological data analysis. 7.) Social Network Analysis. etc.
- 5. Data Clustering Relevant Clustering Algorithms Clustering validation Contents 1 Data Clustering Feature Selection Methods: Distance based Algorithm 2 Relevant Clustering Algorithms K-means algorithm Fuzzy C-means Algorithm Advantages and disadvantages of K-means and Fuzzy C-means Algorithms 3 Clustering validation Dunn and Dunn index Davies Bouldin index
- 6. Data Clustering Relevant Clustering Algorithms Clustering validation Feature Selection Methods: A preprocessing step in which original subsets of features are selected. Needed in order to enhance the quality of underlying clustering. Noisy and irrelevant features are pruned from contention.
- 7. Data Clustering Relevant Clustering Algorithms Clustering validation Contents 1 Data Clustering Feature Selection Methods: Distance based Algorithm 2 Relevant Clustering Algorithms K-means algorithm Fuzzy C-means Algorithm Advantages and disadvantages of K-means and Fuzzy C-means Algorithms 3 Clustering validation Dunn and Dunn index Davies Bouldin index
- 8. Data Clustering Relevant Clustering Algorithms Clustering validation Distance based Algorithm:
- 9. Data Clustering Relevant Clustering Algorithms Clustering validation Contents 1 Data Clustering Feature Selection Methods: Distance based Algorithm 2 Relevant Clustering Algorithms K-means algorithm Fuzzy C-means Algorithm Advantages and disadvantages of K-means and Fuzzy C-means Algorithms 3 Clustering validation Dunn and Dunn index Davies Bouldin index
- 10. Data Clustering Relevant Clustering Algorithms Clustering validation K-means algorithm An unsupervised learning algorithm. Applies on the m-dimensional hyperspace,for a given data set. The pre-processing steps are: ’Handling missing values’ and ’Scaling’. Scaling : If the attribute is A.and have range [Amin, Amax ].Then, to scale a value of A as A x, the formula is: Ax (scaled) = (Ax - Amin)/(Amax - Amin)
- 11. Data Clustering Relevant Clustering Algorithms Clustering validation K-means Algorithm: Handling Missing values: For example: 1.) Replace the missing values by zero(if numerical). 2.) Replace it by the maximum possible value. 3.)Fill in missing values manually based on your domain knowledge. 4.)Replace them with the variable mean (if numerical) or the most frequent value (if categorical). Input: The data set, value of K (number of clusters). Output: The clustered data set (each data element must be assigned to any one the clusters).
- 12. Data Clustering Relevant Clustering Algorithms Clustering validation In steps, the algorithm is as: Step 1.) Initialise the k centroids for k clusters by randomly selecting them as a point in m-dimensional hyperspace.Label them uniquely. Step 2.) For each data element,do : 2.i) Calculate the distances from every cluster centroid. 2.ii) Compare the distances and give the cluster label of each data element as the label of centroid nearest to it. Step 3.) For each cluster,do : Calculate the mean of values of each data element within a cluster.Shift the centroid to the calculated mean in previous step. Step 4.) 4.i) Calculate the change in position of each cluster centroids and add them all. 4.ii) If the sum calculated sum is greater than the pre-specified threshold or the number of iterations is more than the limit,then go to step 2. Step 5.) Terminate.The data set with cluster labels is the result.
- 13. Data Clustering Relevant Clustering Algorithms Clustering validation Example: three-means iris data First iteration:
- 14. Data Clustering Relevant Clustering Algorithms Clustering validation Example: three-means iris data Second iteration:
- 15. Data Clustering Relevant Clustering Algorithms Clustering validation Example: three-means iris data
- 16. Data Clustering Relevant Clustering Algorithms Clustering validation Example: three-means iris data
- 17. Data Clustering Relevant Clustering Algorithms Clustering validation Contents 1 Data Clustering Feature Selection Methods: Distance based Algorithm 2 Relevant Clustering Algorithms K-means algorithm Fuzzy C-means Algorithm Advantages and disadvantages of K-means and Fuzzy C-means Algorithms 3 Clustering validation Dunn and Dunn index Davies Bouldin index
- 18. Data Clustering Relevant Clustering Algorithms Clustering validation Fuzzy C-means Algorithm: Need for fuzzy: In case of overlapping clusters, Hard-clustering is not feasible. Then,to extract such overlapping structures,Fuzzy C-means is used. • Fuzzy c-means allows data points to be assigned into more than one cluster, therefore each data point has a degree of membership (or probability) of belonging to each cluster. Algorithm:Fuzzy Clustering is carried out through an iterative optimization of the objective function: where,
- 19. Data Clustering Relevant Clustering Algorithms Clustering validation Fuzzy C-means Algorithm m is any real number greater than 1 and determines the level of cluster fuzziness. A large m results in smaller memberships wij and hence, fuzzier clusters. wij is the degree of membership of xi in the cluster j, xi is the ith of d-dimensional measured data, cj is the d-dimension center of the cluster, and ||xi − cj || is any norm representing the similarity(or dissimilarity) between any measured data xi and the centroid cj . Input: Data Set X = {x1, x2, x3, ...., xn}, value of C (number of clusters), value for m.
- 20. Data Clustering Relevant Clustering Algorithms Clustering validation Fuzzy C-means Algorithm Output: A set of clusters C = {c1, c2, c3, ...., cn}, A partitioning matrix W as: In steps,the algorithm is as: Step 1.) Initialise the C centroids for C clusters by randomly selecting them as a point in m-dimensional hyperspace.Label them uniquely. Step 2.) For each data element xi , do : 2.i) Calculate the distances(or similarity measure), ||xi − cj || from every cluster centroid. 2.ii) Calculate the fuzzy membership wij , of xi to belong in cj , by:
- 21. Data Clustering Relevant Clustering Algorithms Clustering validation Fuzzy C-means Algorithm And fill the value of wij in matrix W. Step 3.) For each cluster,do : 3.i) Calculate the new position(or value) of centroid,ck by : 3.ii) Shift the centroid to the calculated position(or value) in previous step. Step 4.) 4.i) compute and update values of elements of W (k) as W (k+1 4.ii) If ||W (k+1) − W (k)|| > β, then go to Step 2. Else The present value of W is the resultant partitioning matrix. where, k is the iteration step. β is the termination criterion, β ∈ [0, 1]. ’W (k)’ is the fuzzy membership matrix in kth iteration.
- 22. Data Clustering Relevant Clustering Algorithms Clustering validation Contents 1 Data Clustering Feature Selection Methods: Distance based Algorithm 2 Relevant Clustering Algorithms K-means algorithm Fuzzy C-means Algorithm Advantages and disadvantages of K-means and Fuzzy C-means Algorithms 3 Clustering validation Dunn and Dunn index Davies Bouldin index
- 23. Data Clustering Relevant Clustering Algorithms Clustering validation Advantages and disadvantages of K-means and Fuzzy C-means Algorithms: The K-means algorithm is fast, robust and easier to understand. FCM is more complex that K-means. K-means gives better result when data set are distinct or well separated from each other, that is for non-overlapping clusters.FCM is better for overlapping clusters. The limitations of both is that, the value of K must be known priorly, K-mean results in local optima,it is applicable only when mean is defined, hence fails for categorical data and is unable to handle noisy data and outliers[1] The time complexity of the K-Means algorithm is O(tkdn) and the time complexity of FCM algorithm is O(ndk2t) where, n is number of data objects, k is number clusters, d is dimension of each object and t is iterations. Normally, k, t, d << n[5]
- 24. Data Clustering Relevant Clustering Algorithms Clustering validation How do we know that a particular clustering is good or that it solves the needs of the applications??
- 25. Data Clustering Relevant Clustering Algorithms Clustering validation How do we know that a particular clustering is good or that it solves the needs of the applications?? Given a particular clustering how do we know, what the quality of the clustering really is???
- 26. Data Clustering Relevant Clustering Algorithms Clustering validation Cluster validation Evaluation of clustering results sometimes is referred to as cluster validation.[7] Measures are used to compare the quality of different clustering algorithms. The measures are classified as: Internal Indices. External Indices.
- 27. Data Clustering Relevant Clustering Algorithms Clustering validation Internal Indices: Evaluation is Based on the data that was clustered itself. For Example: Dunn and Dunn index. Davies-Bouldin index. etc.
- 28. Data Clustering Relevant Clustering Algorithms Clustering validation Contents 1 Data Clustering Feature Selection Methods: Distance based Algorithm 2 Relevant Clustering Algorithms K-means algorithm Fuzzy C-means Algorithm Advantages and disadvantages of K-means and Fuzzy C-means Algorithms 3 Clustering validation Dunn and Dunn index Davies Bouldin index
- 29. Data Clustering Relevant Clustering Algorithms Clustering validation Dunn and Dunn index: It is proposed by [3]. Identifies clusters which are well separated and compact. Goal is[6]: to maximize the inter-cluster distance. minimizing the intra-cluster distance. The Dunn index for k clusters is defined by:
- 30. Data Clustering Relevant Clustering Algorithms Clustering validation Dunn and Dunn index: where, is the dissimilarity between clusters ci and cj ; and is the intra-cluster function (or diameter) of the cluster. • If Dunn index is large, it means that compact and well separated clusters exist.
- 31. Data Clustering Relevant Clustering Algorithms Clustering validation Dunn and Dunn index: The Dunn index is: Computationally expensive Sensitive to noisy data Useful for identifying clean clusters in data sets.
- 32. Data Clustering Relevant Clustering Algorithms Clustering validation Contents 1 Data Clustering Feature Selection Methods: Distance based Algorithm 2 Relevant Clustering Algorithms K-means algorithm Fuzzy C-means Algorithm Advantages and disadvantages of K-means and Fuzzy C-means Algorithms 3 Clustering validation Dunn and Dunn index Davies Bouldin index
- 33. Data Clustering Relevant Clustering Algorithms Clustering validation Davies Bouldin index: The Davies Bouldin index [8] is based on similarity measure of clusters (Rij ). Dispersion(si ) of a cluster and dissimilarity between clusters(dij ) are used to compute the Davies-Bouldin(DB) index. Similarity measure of clusters, (Rij ) must satisfy the conditions: Rij >= 0 Rij = Rji if si = 0 and sj = 0 then Rij = 0 if sj >sk and dij = dik then Rij >Rik if sj = sk and dij <dik then Rij >Rik
- 34. Data Clustering Relevant Clustering Algorithms Clustering validation Davies Bouldin index: Usually Rij is defined in the following way[4]: Then the Davies Bouldin index is defined as: where,
- 35. Data Clustering Relevant Clustering Algorithms Clustering validation Davies Bouldin index: The Davies Boludin index measures the average of similarity between each cluster and its most similar one. As the clusters have to be compact and separated, the lower Davies Bouldin index means better cluster configuration. Davies-Bouldin index gives good results for distinct groups. Its not designed to accommodate overlapping clusters.
- 36. Appendix References I [1] site/dataclusteringalgorithms/k-means-clustering-algorithm. [2] Aggrawal, C. C., and Reddy, C. K. Data Clustering. CRC Press. [3] Dunn, J. Well separated clusters and optimal fuzzy partitions. Journal of Cybernetics 4 (1974), 95104. [4] Ferenc Kovcs, Csaba Legny, A. B. Cluster validity measurement techniques.
- 37. Appendix References II [5] Ghosh, S., and Dubey, S. K. Comparative analysis of k-means and fuzzy c-means algorithms. ((IJACSA) International Journal of Advanced Computer Science and Applications 4(4) (2013). [6] Sandro Saitta, B. R., and Smith, I. F. A bounded index for cluster validity. [7] wikipedia. https://en.wikipedia.org/wiki/cluster analysis#evaluation and assessm
- 39. Appendix Thank You! You can contact me at [email protected]