See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/228895611

Genetic algorithm applied in Clustering datasets

Article · January 2002

CITATIONS READS
3 153

1 author:

James Cunha Werner

71 PUBLICATIONS 214 CITATIONS

SEE PROFILE

All content following this page was uploaded by James Cunha Werner on 26 July 2015.

The user has requested enhancement of the downloaded file.

 Genetic algorithm applied in Clustering datasets.

James Cunha Werner ([email protected] )

Terence C. Fogarty ([email protected] )

SCISM
South Bank University
103 Borough Road
London SE1 0AA

Abstract. This paper compares the clustering technique k-means and two
different approaches of Genetic Algorithms to a sample dataset, and in
EachMovie dataset. The comparison between both techniques shows a
better result of GA.

Introduction.

Unsupervised learning is a technique of knowledge extraction from datasets where there is not a correct
solution to compare with the estimated output of the model. The best output or model groups the
elements with some attribute, building a cluster with minimum total distance with a metric definition.
The final result contains a set of elements that are a base for all elements, with minimise the adopted
metrics. All events of one cluster have some attribute in common: kind of movie or book, taste, smell,
diagnostic, etc. This information would help to recommend similar thinks with same characteristics to a
customer, bringing up new business opportunities.
Preparing data for this kind of problem require some non-objective data acquisition. For example,
movie classification would differ for each people between different classes like animation and family,
or sometimes horror and comedy, due different points of view, social class and personality. It’s
difficult to establish a classification for the users too.
To solve this, a large amount of users would be required to establish an average classification for each
product, with a mark of like/dislike or a classification. The next step consist in obtain the sets with
group all events with a minimum distance, the objective of cluster technique.
This paper describes the techniques k-means and genetic algorithm, and its application in one example
for comparative study and the application of genetic algorithms to EachMovie dataset for a proper
performance study.

K-means algorithm.

This algorithm is based in an adaptive approach where a random set of cluster base is select from the
original dataset, and each element update the nearest element of the base with the average of its
attributes (see [1] for a detailed description). To study this algorithm we used an available in Internet
software with an example dataset (see [2]).

Genetic algorithm.

Genetic algorithms (GA) mimics the evolution and improvement of life through reproduction, when
each individual contributes with its own genetic information building a new one with fitness to the
environment and more surviving chances. These are the bases of genetic algorithms and programming
(Holland [3], Goldberg [4] and Koza [5]). Specialized Markov Chains underline the theoretical bases of
this algorithm change of states and searching procedures.
Each ‘individual’ of the generation represents a feasible solution to the problem, coding distinct
algorithms/parameters that should be evaluated by a fitness function.
GA operators are mutation (the change of a random position of the chromosome) and crossover (the
change of slices of chromosome between parents).
The best individuals are continuously being selected, and crossover and mutation take place. Following
few generations, the population converges to the solution that better attend the performance function.
Genetic algorithm concepts and implementation are structured in the following diagram.
 Population initialization

Step 1 : Parent selection.

First population Objective value z=f(x,y)
1100110110101000 3.481746
0101010110110101 3.668023
==> 1000010100110110 6.261380
==> 1101011111001100 12.864222

Step 2: Crossover
11010 11111001100 ------- \ 11010 10100110110
10000 10100110110 ------- / 10000 11111001100

Step 3: Mutation
1101010100110110 1000011111001100

1111010100100110 1000011111001100
z = 8.044 z = 6.092

Step 4: Reinsertion
second population Objective value z=f(x,y)
==> 1111010100100110 8.044
==> 1000011111001100 6.092
1000010100110110 6.261380
1101011111001100 12.864222

Step 5: If the result doesn’t converge, go to step 1.

Comparative example.

The data available with the k-means software consist of the name of the element, and 10 attributes. The
distance is the Euclidian distance between normalized elements.
GA should apply two different approaches:
1. Coding each component of the cluster in the chromosome: in this case, GA must find each
component for each cluster base, resulting a chromosome size with is the number of attributes
times the number of cluster in the base.
2. The chromosome contains for each element the cluster it pertains, and the centre is the average
of all members. The chromosome size is the number of elements.
The fitness function in both cases is:
 1.0
Fitness =
 
∑ min  ∑ ( patterni − cluster j )2 
cluster j  attributei 
mark
The same fitness function could be evaluated for all approaches, and the final value for k-means were
0.138. GA using the number of the cluster for each element (option 2) gave a final fitness of 0.180 and
the case of coding each element (option 1) of the base gave 0.189.
Both cases the GA result show be more accurate then k-means approach.

Genetic algorithm application with EachMovie dataset.

To apply GA in a clustering problem, we used the EachMovie dataset available in Internet [6], with
contains the 1.732.627 votes of 47.636 people for 1.628 movies. The data were filtered remaining the
corrects and completes one, from where we normalise and selected the following attributes:
• Person (person.txt) provides optional, unaudited demographic data supplied by each person:
ID: Number -- primary key
Age: Number
Gender: Text -- one of "M", "F"
• Movie (movie.txt) provides descriptive information about each movie:
ID: Number -- primary key
Name: Text
Type: Action, Animation, Art_Foreign, Classic, Comedy, Drama, Family, Horror,
Romance, Thriller: Yes/No
• Vote (vote.txt) is the actual rating data:
Person_ID: Number
Movie_ID: Number
Mark or Score: Number -- 0 <= Score <= 1
We adopted the approach of determine the cluster base for 40 different groups and 3 attributes: sex,
age, and type. Type is obtained by the association of 10 bits sequence respectively to the movie type,
for example, action is 1 base 2 and 1 base 10, Art_Foreign is 100 base 2 or 4 base 10, etc. Table I
shows the more important groups elements.

Male Female
Cluster: 16 Cluster: 22
Age: 17 Age: 11
Animation, classic, comedy, drama. Animation, Art_Foreign, Classic, Comedy
# Patterns in Cluster: 118190 # Patterns in Cluster: 22619

3568 Batman (1989) 441 Independence Day (ID4)

2819 Toy Story 426 Toy Story
2093 The Silence of the Lambs 411 Twister
1978 Seven 330 Mission: Impossible
1957 Ace Ventura: Pet Detective 312 Mr. Holland's Opus
1806 Star Wars 279 Willy Wonka and the Chocolate Factory
1724 Mrs. Doubtfire 256 The Birdcage
1659 Schindler's List 239 The Truth about Cats and Dogs
1636 The Lion King 231 The Nutty Professor
1553 Batman Forever 221 Grumpier Old Men
 Male Female
Cluster: 10 Cluster: 28
Age: 19 Age: 20
Romance, classic Romance, Art_Foreign, Animation
# Patterns in Cluster: 44328 # Patterns in Cluster: 18553

3888 Aladdin 1625 Aladdin

3031 Beauty and the Beast 1471 Beauty and the Beast
2810 Goldeneye 903 Clear and Present Danger
2713 Outbreak 880 Four Weddings and a Funeral
2612 Clear and Present Danger 875 Outbreak
2395 Heat 775 The River Wild
1820 Addams Family Values 753 Goldeneye
1550 Executive Decision 746 Heat
1322 Four Weddings and a Funeral 725 Addams Family Values
1262 The River Wild 652 Executive Decision
Cluster: 11 Cluster: 38
Age: 21 Age: 21
Action, Animation, Art_Foreign Drama, action
# Patterns in Cluster: 252942 # Patterns in Cluster: 84599

6072 Pulp Fiction 1198 Independence Day (ID4)

5742 True Lies 1108 Batman (1989)
5122 12 Monkeys 1037 Toy Story
5004 The Fugitive 1009 The Birdcage
4585 Mission: Impossible 933 Dances With Wolves
4315 Twister 893 Twister
4229 Independence Day (ID4) 880 The Lion King
3956 Broken Arrow 862 Mission: Impossible
3689 The Rock 821 The Silence of the Lambs
3542 Speed 820 12 Monkeys
Cluster: 14 Cluster: 17
Age: 28 Age: 24
Drama, comedy, Art_Foreign, Animation Thriller, Drama, Animation action
# Patterns in Cluster: 156129 # Patterns in Cluster: 19861

6025 Batman (1989) 1512 Jurassic Park

4413 The Silence of the Lambs 1436 Father of the Bride Part II
4028 Toy Story 1291 The Shawshank Redemption
3068 Seven 945 Terminator 2: Judgment Day
2948 Dances With Wolves 839 Crimson Tide
2876 Star Wars 836 Cliffhanger
2610 Mrs. Doubtfire 696 The Santa Clause
2600 Schindler's List 658 Apollo 13
2411 Ace Ventura: Pet Detect ive 602 Multiplicity
2389 Dead Man Walking 557 101 Dalmatians (1996)
Cluster: 25 Cluster: 6
Age: 28 Age: 28
Romance, comedy Comedy, Art_Foreign, Animation, Action
# Patterns in Cluster: 54527 # Patterns in Cluster: 79905

4484 Clear and Present Danger 1046 Mission: Impossible

4281 Aladdin 1042 Twister
3509 Beauty and the Beast 1009 Independence Day (ID4)
3087 Goldeneye 979 Dances With Wolves
3009 Outbreak 974 Pulp Fiction
2631 Four Weddings and a Funeral 952 12 Monkeys
2524 Heat 922 The Fugitive
2173 Executive Decision 919 The Birdcage
2078 Addams Family Values 917 True Lies
1584 Philadelphia 879 Toy Story
 Male Female
Cluster: 2 Cluster: 30
Age: 32 Age: 37
Action, classic, comedy Horror, family, drama, comedy, animation
# Patterns in Cluster: 229666 # Patterns in Cluster: 10974

4445 Pulp Fiction 848 Aladdin

4368 True Lies 842 Beauty and the Beast
3812 The Fugitive 669 Clear and Present Danger
3704 Dances With Wolves 565 Outbreak
3222 12 Monkeys 553 The River Wild
3165 Independence Day (ID4) 550 Four Weddings and a Funeral
3160 Mission: Impossible 521 Executive Decision
3145 Twister 445 Heat
2682 Broken Arrow 402 Goldeneye
2565 Die Hard: With a Vengeance 386 Addams Family Values
Cluster: 19 Cluster: 32
Age: 43 Age: 39
Comedy, classic, animation Drama, Art_Foreign, Animati on
# Patterns in Cluster: 153114 # Patterns in Cluster: 61588

2699 True Lies 898 Independence Day (ID4)

2580 Dances With Wolves 844 Toy Story
2568 The Fugitive 834 Dances With Wolves
2268 Pulp Fiction 825 Twister
2182 Independence Day (ID4) 772 The Birdcage
2163 Batman (1989) 764 Batman (1989)
2132 Twister 727 Mr. Holland's Opus
1900 Mission: Impossible 686 Mission: Impossible
1834 Forrest Gump 636 Forrest Gump
1831 12 Monkeys 632 Sense and Sensibility
Cluster: 7 Cluster: 40
Age: 49 Age: 43
Drama, comedy, classic, action Romance, horror, family, drama, art_foreign, action
# Patterns in Cluster: 58880 # Patterns in Cluster: 10000

1742 Batman (1989) 897 Jurassic Park

1485 The Lion King 865 The Shawshank Redemption
1284 The Silence of the Lambs 792 Father of the Bride Part II
1239 Schindler's List 544 Crimson Tide
1195 Dances With Wolves 475 Cliffhanger
1057 Toy Story 423 The Santa Clause
1046 Mr. Holland's Opus 370 Terminator 2: Judgment Day
940 Mrs. Doubtfire 355 Multiplicity
937 Dead Man Walking 281 101 Dalmatians (1996)
852 Star Wars 239 Beverly Hills Cop III
Cluster: 29 Cluster: 34
Age: 55 Age: 50
Comedy, classics, action Romance, classic, animation, action
# Patterns in Cluster: 56495 # Patterns in Cluster: 4797

1036 Dances With Wolves 354 Clear and Present Danger

1006 The Fugitive 340 Aladdin
979 True Lies 311 The River Wild
974 Pulp Fiction 294 Executive Decision
948 Twister 291 Beauty and the Beast
864 Mission: Impossible 289 Four Weddings and a Funeral
844 Independence Day (ID4) 241 Outbreak
715 Fargo 202 Philadelphia
707 Sense and Sensibility 187 Heat
700 Braveheart 174 Addams Family Values

The least initial distance of the first generation was 62,321, and the final result after converge was
7,098. The result shows that if I (man with 43 years old) am in Blockbuster lending “Dances with
wolves”, should be invited to lend “The fugitive” and “True lies”, which in my opinion is satisfactory.

Conclusion

A class of problems without a training dataset, where the relation between excitation and answer is
formally defined, can be addressed by clustering algorithms.
 Working with clustering means define a metric distance and the features for what each event will be
grouped to obtain the least distance for all set. In EachMovie application, the definition of type of
movie as 10 bits adjacent is a feature definition that would affect the solution because, for example,
there is a distance between drama (32 base 10 or 100000 base 2) and drama and action (33 base 10 or
100001 base 2) that is not the same (or comparable) as animation (2 base 10) and action (1base 10).
The definition of distance as Euclidian affect this position as well.
The application of genetic algorithms to clustering problems shows better results than k-means
approach, under a controlled example, and the result of the clustering for different movies are
reasonable for the author enjoyment.
However, the classification, the data acquisition and the clusters are completely subjective: there is not
a correct answer. There is not a function able to classify the same products to any individual, but it
would recommend other alternatives and new business opportunities would be created.

References.

1. Bow,S.T; “Pattern recognition applications to large dataset problems”

2. Kyoung-june Min ; http://vorlon.cwru.edu/~kxm73/ and
http://vorlon.cwru.edu/~kxm73/CW/eces484/eces484.html
3. HOLLAND,J.H. “Adaptation in natural and artificial systems: an introductory analysis
with applications to biology, control and artificial intelligence.” Cambridge: Cambridge
press 1992 reedição 1975.
4. GOLDBERG,D.E. “Genetic Algorithms in Search, Optimisation, and Machine Learning.”
Reading,Mass.: Addison-Whesley, 1989.
5. KOZA,J.R. “Genetic programming: On the programming of computers by means of
natural selection.” Cambridge,Mass.: MIT Press, 1992.
6. Steve Glassman; [email protected] Compaq Systems Research
Center 130 Lytton Avenue Palo Alto, CA 94301

View publication stats