Survey on Text Mining Based on Social Media Comments as Big Data Analysis Using Genetic Algorithm

International Journal of Modern Research in Engineering & Management (IJMREM)
||Volume|| 1||Issue|| 9 ||Pages|| 37-46 || October 2018|| ISSN: 2581-4540
www.ijmrem.com IJMREM Page 37
Survey on Text Mining Based on Social Media Comments as Big
Data Analysis Using Genetic Algorithm
1,
Brijesh Kumar, 2,
Ankita Khan, 3,
Prof. Ramya R, 4,
Prof. Divyashree G
1,2,
5th
Semester, I.S.E , Sapthagiri College of Engineering
3,4,
Assistant Professor, I.S.E Sapthagiri College of Engineering
-----------------------------------------------------ABSTRACT -----------------------------------------------------
The tax gives an important role for the contributions of the economy and development of a country. The
improvements to the taxation service system continuously done in order to increase the State Budget. The
performance of the country will be upgrade from the public opinion about the tax. The opinion of the public will
be considered as a data for the growth of the nation. Text mining can be used to know public opinion about the
tax system. The rapid growth of data in social media initiates the researchers to use the data source as big data
analysis. The dataset used is derived from Face book, Twitter public sentiment in part of service, website
system, and news can be used as consideration as a input of tax comments. In this paper, text mining is done
through the phases of text processing, feature selection and classification with genetic algorithm (GA). Efficient
framework is used for pre-processing the data. Testing is used to measure the performance level of GA by using
the evaluation metrics such as purity, entropy and F-measure.
KEYWORDS: Big data, text mining, genetic algorithm, tax comments, data pre-processing.
----------------------------------------------------------------------------------------------------------------------------------------
Date of Submission: Date,04 October 2018 Date of Publication: Date 15. October 2018
---------------------------------------------------------------------------------------------------------------------------------------
I. INTRODUCTION
Data mining is known as Knowledge Discovery in Databases (KDD).Data mining is a process of analyzing large
databases to find patterns that are valid, useful, and understandable. The valid means holds the new data with
some certainty and useful means data mining should be able to act on the terms in the document finally, the
understandable means humans should be able to read/identify the pattern. Data mining performed with large
data, heterogeneous machine learning, statistics, artificial intelligence, databases and visualization.Text mining
is a part of data mining its aim is to extract high-quality information from the given text. The extraction of high
quality information can be done through statistical pattern learning. text mining includes information retrieval,
lexical analysis, pattern recognition, information extraction, data mining techniques, association analysis,
visualization, and predictive analytics. Cluster analysis or clustering is the process of grouping a set of objects in
such a way that objects which are more similar are grouped under single clusters and the objects which are not
similar are grouped under other clusters. It is the main task of investigating data mining, and a common
technique for statistical data analysis, used in many fields which includes various research fields such
as machine learning, pattern recognition, image analysis, information retrieval, and bioinformatics. Text
(document) clustering is a technique, used to group the documents automatically. It is useful in organizing the
documents based on indexing, that is helpful for sorting and quick searching.
Example:
• Apple can be clustered into the categories of fruits, as well as mobile companies
• An email received by a company, with subject line containing problem, can be parsed into a separate folder,
and to be addressed by the customer care.
Text clustering is a document organization, extraction of the terms and fast information retrieval or filtering.
Document clustering uses the concept of identifiers and identifier extraction. Identifiers are sets of words that
describe the contents within the cluster. Congregating is an example of unsupervised classification [1].
Classification of the data is done based on the procedure that assigns data objects to a set of classes.
Unsupervised means clustering does not depend on any predefined classes and data training examples while
classifying the data objects. Clustering is a crucial area of research, which finds applications in many fields
including bioinformatics, pattern recognition, image processing, marketing, data mining, economics, etc. An
example of document clustering is web document clustering for data searching by users. The application of
document clustering can be classified into two types, online and offline.
Online applications are constrained by efficiency problems when compared offline applications.

Survey on Text Mining Based on Social Media…
Text Clustering is a troublesome task and it is being studied from lustrums but still it is far from a
inconsequential and resolved problem. The some of the challenges among them are:
1. Identifying and choosing suitable features of the text that should be used for clustering.
2. Identifying and choosing proper and similar measures between the texts.
3. Identifying and choosing suitable clustering method as well as utilizing the proper and similar measures.
4. Executing the cluster algorithm in an effective and efficient way making it convenient with respect to
memory as well as CPU resources.
5. Uncovering methods to determine the quality of clustering.
Text document clustering plays an important role in providing intuitive navigation and browsing mechanisms by
organizing large amounts of information into a small number of meaningful clusters. However, the bag of words
representation used for these clustering methods is often unsatisfactory as it ignores relationships between
important terms that do not co-occur literally. However, in spite of a long tradition of research in similarity-
based text document retrieval there is no clustering method that could function as a best to this end. The reason
to stem from the fact that text document clustering must simultaneously deal with quite a number of problems
1. Problem of efficiency: Text document clustering must be efficient because it should be able to do
clustering on ad-hoc collections of documents, e.g. ones found by a search engine through keyword search.
2. Problem of effectiveness: Text document clustering must be effective, i.e., it should relate documents that
talk about the same or a similar domain.
3. Problem of explanatory power: Text document clustering should be able to explain to the user why a
particular result was constructed. Lack of understandability may pose a much bigger threat to the success of
an application that employs text document clustering than a few percentage points decrease in accuracy.
4. Problem of user interaction and subjectivity: Applications that employ text document clustering must be
able to involve the user. The results should be focusing one’s attention on particularly relevant subjects. For
example a search for “health” might turn up food-related issues that a user might want to explore in details
relevant for him, such as “fruits”, “meat”, “vegetables” and others.
The rest of the paper describes as follows: Literature Survey describes some of the current knowledge related to
the text document clustering as well as theoretical and methodological contributions to a clustering method, the
Methodology brief about the systematic, theoretical analysis of the methods applied to a document clustering,
the architecture of the system will brief the overall work of the system. Then it describes the document pre-
processing module. The similarity measures are calculated for the documents are explained in the similarity
measure module. brief about the text comment clustering by using the Genetic algorithm.
II. LITERATURE SURVEY
Cluster analysis is the basic data analysis tools in data mining. Cluster analysis can be used as a standalone data
mining tool for the data distribution and data preprocessing step for data mining algorithms operating on the
clustering of the documents. Clustering algorithms are used to organize data, categorize data, for data
compression and model construction. Clustering is a huge area of research, which finds applications in many
fields including bioinformatics, pattern recognition, image processing, marketing, data mining, economics etc.
Clustering is a group of set of objects and finds the relationship between the objects. Hence the clustering
algorithm and similarity measures are used for the text document clustering. Data clustering algorithms can be
mainly classified into following categories [2]:
1. Partition algorithms : A partition clustering algorithm partition the data set into desired number of sets in
a single step. Partition clustering algorithm [13] splits the data points into k partition, where each partition
represents a cluster. The partition is done based on certain objective function. The partition algorithm
approaches require selecting a value for the desired number of clusters to be generated. Some of the examples
for the partition clustering algorithms methods are k-means and a variant of k-means-bisecting, k-medoids.
2. Hierarchical algorithms : A hierarchical clustering algorithm divides the given data set into smaller
subsets in hierarchical fashion. A hierarchical method [13] creates a hierarchical decomposition of the given set
of data objects. Here tree of clusters called as dendrograms is built. Every cluster node contains child clusters,
sibling clusters partition the points covered by their common parent. In hierarchical clustering we assign each
item to a cluster such that if we have N items then we have N clusters. Find closest pair of clusters and merge

them into single cluster. Compute distance between new cluster and each of old clusters. We have to repeat
these steps until all items are clustered into K no. of clusters. It is of two types:
1. Agglomerative (bottom up): Agglomerative hierarchical clustering is a clustering method where clusters
are divided into sub-clusters, which in turn are divided into sub-clusters, etc. It begins with letting each
object form its own cluster and iteratively merges cluster into larger and larger clusters, until all the objects
are in a single cluster or certain termination condition is satisfied. The single cluster becomes the
hierarchy’s root. For the merging step, it finds the two clusters that are closest to each other, and combines
the two to form one cluster.
2. Divisive (top down) : A top-down clustering method is not a preferred method. It works same as the
agglomerative clustering algorithm but in the reverse direction. This method starts with a single cluster
containing all data, and then in an uninterrupt sequence it splits the resulting clusters unless the clusters of
discrete objects remain. K-Means algorithm has biggest advantage of clustering large data sets and its
performance increases as number of clusters increases. But its use is limited to numeric values. Therefore
Agglomerative and Divisive Hierarchical algorithm was adopted for categorical data, but due to its
complexity a new approach for assigning rank value to each categorical attribute using K- means can be
used in which categorical data is first converted into numeric by assigning rank. Hence performance of K-
mean algorithm is better than Hierarchical Clustering Algorithm.
3. Density-based methods: Density-based clustering methods will group the data objects based on the
arbitrary shapes. Clustering is done based on criteria of density such as density connected points and based on
an explicitly constructed density function. The popular density-based clustering methods are DBSCAN and its
extension, OPTICS and DENCLUE
4. Grid-based methods: Grid-based clustering methods use multiresolution grid structure to cluster the data
objects. The benefit of this method is its speed in processing time. Some examples include STING, Wave
Cluster.
5. Model-based methods: Model-based methods use a model for each cluster and determine the fit of the data
to the given model. It is also used to automatically determine the number of clusters. Expectation-Maximization,
COBWEB and SOM (Self-Organizing Map) are typical examples of model-based methods.
Clustering applications : Clustering is a important and widely used tool in a number of applications and in
various fields of business and science.These are the following information that will summarize the concept used
in clustering.
a. Uncovering Suitable Documents: This technique is mostly used when the client has found one “good”
document in a search analysis and wanted more-like-this. The most fascinating feature of this technique is that
clustering is able to find documents that are similar in contrast to search-based methods that are only able to
find whether the documents contains synonyms of words .
b. Organizing Huge Document Collections: Document redeem focuses on searching documents related to a
particular query ,however it fails to resolve the problem of making sense of a huge amount of unclassified
documents. The challenge is to arrange these documents in a taxonomy which exhibits duplicacy to the one
which users would create, given enough time and use it as a browser which is interacting to the real world
collection of documents.
c. Identical Data Detection: Document redeem focuses on searching documents related to a particular query
,however it fails to resolve the problem of making sense of a huge amount of unclassified documents. The
challenge is to arrange these documents in a taxonomy which exhibits duplicacy to the one which users would
create, given enough time and use it as a browser which is interacting to the real world collection of documents.
d. Recommended System: Using this system, the client is recommended articles and depending on the articles
that the client has already read, clustering of the recommended articles are successful in real time which in turn
improves their quality.
e. Finding The Best Available Resource: Clustering assists inorder to improve the quality and efficiency of
search engines. In this, the query of the client can be initially compared to the clusters irrespective of comparing
it directly to the texts and also the arrangement of the search results can be arranged with ease.

Prior Studies ; This section describes abstraction of few of the related work, in terms of the problem taken in
hand, their approach and finding.
Mohit Sharma and Pranjal Singh “Text mining in big data and Similarity Measures” [16] Mining is an
important technique which organizes a large number of objects into small number of coherent groups. It leads to
efficient and effective use of these text for information retrieval. Clustering algorithms require a similarity
metric to identify how the two different textare related/similar to each other. This difference is often measured
by some distance measure such as Cosine similarity, jaccard and others. In the work, the well known five
different distance measures are used and compare their performance on datasets using k-means clustering
algorithm. But, this work has a lack of efficient feature selection and representing the terms.
Anna Huang, “Similarity Measures for Text Document Clustering” [14] Clustering is a useful technique
that groups a large quantity of unordered text documents into a small number of meaningful and coherent
clusters. Partitional clustering algorithms have been identified more a more suitable than the hierarchical
clustering algorithm schemes for clustering the large datasets. The different types of similarity measures have
been used for clustering the data, such as euclidean distance measure, cosine similarity, and relative entropy. In
the paper, different similarities measures are used to compared and analyze their effectiveness by using the
similarity measures in partitional clustering for text document datasets. The work utilize the standard K-means
clustering algorithm and report the results on seven text document datasets and five distance/similarity measures
that have been most commonly used in text clustering. In the observed work there are three components that
affect the final results, they are representation of the terms, distance or similarity measures, and the clustering
algorithm itself.
Mihuandayani, “Text mining based on tax comments as big data analysis using SVM and feature
selection”[1] The paper discussed The dataset used is derived from Facebook and Twitter as a source of data in
processing tax comments. The results of opinions in the form of public sentiment in part of service, website
system, and news can be used as consideration to improve the quality of tax services. In this research, text
mining is done through the phases of text processing, feature selection and classification with Support Vector
Machine (SVM). To reduce the problem of the number of attributes on the dataset in classifying text, Feature
Selection used the Information Gain to select the relevant terms to the tax topic. Testing is used to measure the
performance level of SVM with Feature Selection from two data sources. Performance measured using the
parameters of precision, recall, and F-measure.
Gaps identified
➢ The work has a lack of efficient feature selection and
Representing the terms.
➢ SVM don’t perform the iterative process to identify the positive and negative comments.
➢ To design and develop the efficient text mining based on social media comments by using genetic algorithm
technique and evaluate its performance
III EXIXTING SYSTEM
The tax gives an important role for the contributions of the economy and development of a country. The
improvements to the taxation service system continuously done in order to increase the State Budget. One of
consideration to know the performance of taxation articularly in Indonesia is to know the public opinion as for
the object service. Text mining can be used to know public opinion about the tax system. The rapid growth of
data in social media initiates this research to use the data source as big data analysis. The dataset used is derived
from Facebook and Twitter as a source of data in processing tax comments. The results of opinions in the form
of public sentiment in part of service, website system, and news can be used as consideration to improve the
quality of tax services. In this research, text mining is done through the phases of text processing, feature
selection and classification with Support Vector Machine (SVM). To reduce the problem of the number of
attributes on the dataset in classifying text, Feature Selection used the Information Gain to select the relevant
terms to the tax topic. Testing is used to measure the performance level of SVM with Feature Selection from
two data sources. Performance measured using the parameters of precision, recall, and F- measure.
The below figure1 shows that the exixting system text mining flowchart. the next phase of selecting the relevant
features is the classification phase using SVM algorithm which purposed to find the function of the separator
(hyperplane) with the largest margin, so as to separate the two data sets optimally [19]. The SVM algorithm is
very effective used to handle the text classification problem [20]. SVM is capable of working on high-

dimensional datasets using the trick kernel. SVM uses only some of the selected data points that contribute
(Support Vector) to form the model used in the classification process. In this study, SVM was used to find
hyperplane that separated the data on positive comments and negative comments. SVM used the kernel to
transform the input into the feature space or implement the model to a higher dimension so that the nonlinear
case separable on the input becomes linear separable on a feature space.
Figure 1: Existing system text mining flowchart
IV. PROPOSED WORK
Figure 2 shows the overall methodology of the proposed efficient document clustering system on centralized
system.
The different steps in the architecture are defined as following,
➢ Only tax comment datasets are collected from the social media like face book, twitter, news etc.
➢ The collected tax comments are pre-processed using different techniques such as removal of stop words and
stemming the words in each system. Stop words are the non-descriptive words such as a, and, are, then,
what, is, the, do etc. are removed from the each of the comment. Stemming the words means words with
different endings will be mapped into a single word Ex: production, produce, product, produces will be
mapped to the stem “produc” to reduce the appearance of same words with different forms. After pre-
processing steps are performed now, the data are ready to calculate the similarity measures.
➢ Initially calculate the term frequency (TF) and inverse document frequency (IDF) for each comment. Term
frequency tells occurrence of the word in the comment and inverse document frequency represents the
weight of word means to tell how important the word in the comment.
➢ The similarity measures such as Cosine similarity, Jaccard coefficient and Pearson correlation coefficient is
applied to the pre-processed comment in the datasets.
➢ Apply the Genetic algorithm to each of the calculated similarity values at initial level then it produces the
two clusters based on the similarity value that is positive comment cluster and negative comment cluster.
➢ The clustered data are received by apply Genetic algorithm for the tax comment clustering. Genetic
algorithm apply the step by step process and perform the clustering operation on the text documents

➢ Performance factors/metrics such as purity, entropy and F-Measure are used to analyze the proposed
document clustering. Quality of the cluster is calculated by purity and entropy whereas the accuracy of the
cluster is calculated by F-Measure.
Figure 2: Proposed system Architecture
The architecture of the document clustering consists of following five modules
1. Document Pre-processing Module
2. Similarity Measures Module
3. Implementing genetic algorithm
4. Evaluation Metric Module
Document Pre-processing Module
The text document pre-processing can be done by the following process shown in the figure 4.1.
Text document collection: Selecting and accessing the data from the system to perform the document
clustering. The collected document should be in the .txt format.
Text Document Preprocessing: Initially the collected text documents are composed of a lot of elements or the
words. Preprocessing requires the reduction in the document contents.
Figure 3: Text document pre-processing

Text document Collection: includes the processing of data like indexing, filtering etc. These are applied in
collecting the documents which are to be clustered and they are indexed to retrieve and store in an effective
way. It is also used in filtering inorder to eradicate the excess data, like, stop words.
Removal of Stop Word: Stop words are the words that are non-descriptive for the topic of a document such as
a, and, are, then, what, is, the, do etc. It is the frequently occurring words that are not searchable. This is done to
improve the speed and memory consumption of the application. There are standard stop word lists available but
in most of the applications these are modified depending on the quality of the dataset. The syntax to remove the
stop words is Stemming the words means words with different endings will be mapped into a single word or
Stemming is the process of reducing words to their stem or root form. For example ‘learn’, ’learning’, ‘learned’
are forms of the same word used in different constraint for computing similarity. They are considered same and
production, produce, product, produces will be mapped to the stem “produc”.
Preprocessed data preprocessing consists of steps that take as input a plain text document and output a set of
tokens to be included in the vector model.
TF: Term Frequency, which measures how frequently a term occurs in a document. Since every document is
different in length, it is possible that a term would appear much more times in long documents than shorter ones.
Thus, the term frequency is often divided by the document length (aka. the total number of terms in the
document) as a way of normalization:
TF (t) = (Number of times term t appears in a document) /
(Total number of terms in the document)
IDF: Inverse Document Frequency, which measures how important a term is. While computing TF, all terms
are considered equally important. However, it is known that certain terms, such as "is", "of", and "that", may
appear a lot of times but have little importance. Thus, we need to weigh down the frequent terms while scale up
the rare ones, by computing the following:
IDF (t) = loge (Total number of documents / Number of
documents with term t in it)
Similarity Measures Module : A similarity measure or similarity function is a real valued function that
quantifies the similarity between two objects. The similarity measure gives the degree up to which each objects
are close to or separate from each other. This module performs the calculation of the Cosine similarity, Jaccard
coefficient and Pearson correlation coefficient
A variety of similarity or distance measures have been proposed and widely applied, such as the cosine
similarity, Jaccard coefficient and Pearson correlation coefficient.
Cosine Similarity : The similarity of two documents corresponds to the correlation between the vectors, where
the documents are represented as term vectors. This is quantified as the cosine of the angle between vectors,
which is called the cosine similarity. An important property of the cosine similarity is its independence of
document length .The result of the cosine similarity lies between 0-1. If the cosine similarity between the two
documents is 1 then, the documents are similar. If the cosine similarity between the two documents is 0 then, the
documents are not similar. The mathematical formula to calculate the cosine similarity is given as shown in
equation
Where, A- Term count value in
the document1
B- Term count value in the document2
Jaccard Coefficient: The Jaccard coefficient, also known as Tanimoto coefficient, measures similarity as the
intersection divided by the union of the objects. For text document, the Jaccard coefficient compares the sum
weight of shared terms to the sum weight of terms that are present in either of the two documents but are not the
shared terms. The value of the Jaccard coefficient exists in the range of 0-1.The value 1 means the objects are
similar and the value 0 means the documents are different. The formal definition is in the equation
Where, A- Term count value in the document1
B- Term count value in the document2

Pearson correlation coefficient : Pearson’s correlation coefficient is another measure of the extent to which
two vectors are related. There are different forms of the Pearson correlation coefficient formula. The value of
this measure lies between the 0 to 1. The value is 1 when the number of terms present in the document1 is equal
to the number of terms present in the document2. Given the term set T = {t1, . . . , tm}, a commonly used in the
form is equation
Where, xi – Term count value of the document1
yi – Term count value of the document2
x¯ - Mean term count value of the document1
y¯ - Mean term count value of the document2
Implementing Genetic algorithm : Genetic algorithm developed by Goldberg was inspired by Darwin's theory
of evolution which states that the survival of an organism is affected by rule "the strongest species that
survives". Darwin also stated that the survival of an organism can be maintained through the process of
reproduction, crossover and mutation. Darwin's concept of evolution is then adapted to computational algorithm
to find solution to a problem called objective function in natural fashion. A solution generated by genetic
algorithm is called a chromosome, while collection of chromosome is referred as a population. A chromosome
is composed from genes and its value can be either numerical, binary, symbols or characters depending on the
problem want to be solved. These chromosomes will undergo a process called fitness function to measure the
suitability of solution generated by GA with problem. Some chromosomes in population will mate through
process called crossover thus producing new chromosomes named offspring which its genes composition are the
combination of their parent. In a generation, a few chromosomes will also mutation in their gene. The number of
chromosomes which will undergo crossover and mutation is controlled by crossover rate and mutation rate
value. Chromosome in the population that will maintain for the next generation will be selected based on
Darwinian evolution rule, the chromosome which has higher fitness value will have greater probability of being
selected again in the next generation. After several generations, the chromosome value will converges to a
certain value which is the best solution for the problem. The genetic algorithm is used in various application
areas.
The operations that performs in the genetic algorithm contains are Selection, Crossover and Mutation
1.Selection: The candidate individuals are chosen from the population in the current generation based on their
fitness. The individuals with higher fitness values are more likely to be selected as the individuals of population
in the next generation.
2. Crossover: Crossover is a genetic operator that combines (mates) two chromosomes (parents) to produce a
new chromosome (offspring). The idea behind crossover is that the new chromosome may be better than both of
the parents if it takes the best characteristics from each of the parents. Crossover occurs during evolution
according to a user-definable crossover probability. Consider the following 2 parents which have been selected
for crossover. The “|” symbol indicates the randomly chosen crossover point.
Parent1:11001|010
Parent2:00100|111
After interchanging the parent chromosomes at the crossover point, the following offspring are produced:
Offspring1:11001|111
Offspring2:00100|010
3. Mutation: The mutation operator is applied to each bit of an individual with a probability of mutation rate.
After mutation, a bit that was “0” changes to “1” and vice versa. In fact, it is possible that a regular node
becomes a cluster head and a cluster head becomes a regular node. Individual before mutation: 0 1 1 1 0 0 1 1 0
1 0 individual after mutation: 0 1 1 0 0 0 1 1 0 1 0
D. Algorithm : The Algorithm for the genetic algorithm process is as follows
INPUT: k-means clustered data
OUTPUT: clusters of document
Begin

1. Determine the number of chromosomes, generation, and mutation rate and crossover rate value
2. Generate chromosome and the initialization value with a random value
3. Process steps 4-7 until the number of generations is met
4. Evaluation of fitness value of chromosomes by calculating objective function
5. Chromosomes selection
6. Crossover
7. Mutation
8. New Chromosomes (Offspring)
9. Solution (Best Chromosomes)
End
V EVALUATION METRICS
In order to check the quality and accuracy of the clustering algorithm the proposed system uses the metrics such
as purity, entropy and F-Measure. The purity and entropy measures are used to calculate the quality of the
clusters whereas the F-Measure used to check the accuracy of the clustering operations. The evaluation metrics
such as the purity, entropy and F-measure are explained below
1. Purity: The metric purity evaluates the consistency of a cluster that is the degree to which a cluster contains
documents from a single category. If the purity value is one it contains documents from a single category
therefore it is an ideal cluster. The purity value lies in the range of 0-1.The higher the purity value, better
the quality of clusters. If the purity value is one it contains documents from a single category therefore it is
an ideal cluster. The higher the purity value, better the quality of clusters. The formal definition of purity is
as given below in the equation 8.1
x h (nj
h
) (8.1) Where, max h(nj
h
) - is the number of documents that are
from
the dominant category in cluster Cj and
(nj
h
) - represents the number of documents from cluster
Cj assigned to category h
2. Entropy: In general, is a measure of the number of specific ways in which a system is arranged. This
measure evaluates the distribution of categories in a given cluster. The entropy results lies between the 0-1.
If the entropy value is smaller, the quality of clusters is better. The mathematical formula for the entropy is
given as below in the equation 8.2 (8.2)
Where, (ni
h
) - represents the number of documents in cluster Ci assigned to category h
ni – represents the size of the cluster.
3. F-Measure: It is a combined value of Precision and Recall. The Precision and recall computed for each
class and its weighted average gives the value of F-measure. The value of this metrics also lies between 0-1.
More the F-measure more the accuracy. It is calculated as shown in the equation 8.3
F-Measure= 2 * Precision*Rec all/ (Precision + Recall) (8.3)
Precision and recall are the basic measures used in evaluating strategies in finding the F-Measure evaluation
metric. Both precision and recall is applied to the collected documents and the documents can be assumed as
either relevant or irrelevant data that is it measures the degree of relevancy.
Precision is the ratio of the number of relevant text documents retrieved to the total number of
irrelevant and relevant documents retrieved. It is usually expressed as shown in the equation 8.4
Precision= A/ (A+C) (8.4)
Where, A – Number of relevant documents retrieved
C – Number of irrelevant documents retrieved
Recall is the ratio of the number of relevant records retrieved to the total number of relevant records in the
database. It is usually expressed as shown in the 8.5
Recall = A/ (A+B) (8.5)

Where, A – Number of relevant documents retrieved
B – Number of irrelevant documents retrieved
VI CONCLUSION
In the existing system by using SVM algorithm, the performance evaluation that is the precision, recall and F-
measure average of all dataset were 75 %, 70% and 72% achieved. In Proposed work by using Genetic
algorithm the performance evaluation like purity, entropy, precision, recall and F-measure will produce the
better results than SVM
VII FUTURE ENHANCEMENT
The technology used in tax comment clustering will give the better results as per the survey and the results of
many papers. The genetic algorithms used in this paper still be implemented by using any of the optimized
techniques.
REFERENCES
[1] Mihuandayani, “Text mining based on tax comments as big data analysis using SVM and feature
selection” 2018 ICOIACT
[2] Manjot Kaur, Navjot Kaur, “Web Document Clustering Approaches Using K-Means Algorithm”,
International Journal of Advanced Research in Computer Science and Software Engineering, Volume
3, Issue 5, May 2013.
[3] Andreas Hotho, Steffen Staab, Gerd Stumme, “Text Clustering Based on Background Knowledge”,
Technical Report, volume 425. University of Karlsruhe, Institute AIFB, (2003).
[4] M. Eisenhardt, W. Muller, and A. Henrich, “Classifying documents by distributed P2P clustering.” in
INFORMATIK, 2003.
[5] S. Datta, C. R. Giannella, and H. Kargupta, “Kmeans Clustering over a Large, Dynamic Network,”
Proc. SIAM Int’l Conf. Data Mining (SDM), 2006.
[6] M. Steinbach, G. Karypis, and V. Kumar, “A Comparison of Document Clustering Techniques,” Proc.
KDD Workshop Text Mining, 2000.
[7] G. Forman and B. Zhang, “Distributed Data Clustering Can Be Efficient and Exact,” SIGKDD
Explorations Newsletter, vol. 2, no. 2, pp. 34-38, 2000.
[8] S. Datta, K. Bhaduri, C. R. Giannella, R. Wolff and H. Kargupta, “ Distributed data mining in Peer-to-
Peer network’s”, IEEE Internet Computing, vol.10 , no. 4, pp. 18-26, July 2006.
[9] I. Stoica, R. Morris, D. Karger, F. Kaashoek, and H. Balakrishnan, “Chord: A Scalable Peer-to-Peer
Lookup Service for Internet Applications,” Proc. SIGCOMM, 2001.
[10] Neethi Narayanan, J.E.Judith, Dr.J.Jayakumari “Enhanced Distributed Document Clustering Algorithm
Using Different Similarity Measures”Proc of 2013 IEEE Conf on ICT 2013.
[11] Denny Hermawanto,”Genetic Algorithm for Solving Simple Mathematical Equality Problem”,
Indonesian Institute of Sciences (LIPI), INDONESIA.
[12] A. K. Santra, C. Josephine Christy, “Algorithm and Confusion Matrix for Document Clustering”
Dean, CARE School of Computer Applications, India.
[13] Aastha Joshi, Rajneet Kaur, “Comparative Study of Various Clustering Techniques in Data Mining”
Sri Guru Granth Sahib World University, Fatehgarh Sahib, Punjab, India.
[14] Anna Huang, “Similarity Measures for Text Document Clustering” The University of Waikato,
Hamilton, New Zealand, April 2008.
[15] Sanjivani Tushar Deokar , “Text Documents clustering using K Means Algorithm” International
Journal of Technology and Engineering Science [IJTES], July 2013.
[16] Mohit Sharma and Pranjal Singh “Text Document Clustering and Similarity Measures” IIT Kanpur,
India, November 19, 2013

Survey on Text Mining Based on Social Media Comments as Big Data Analysis Using Genetic Algorithm

Recommended

More Related Content

What's hot (20)

Similar to Survey on Text Mining Based on Social Media Comments as Big Data Analysis Using Genetic Algorithm (20)

More from IJMREMJournal (20)

Recently uploaded (20)

Survey on Text Mining Based on Social Media Comments as Big Data Analysis Using Genetic Algorithm