Lecture 1

Introduction to Data Mining

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 1

This lesson includes:

o 1.1 Why Data Mining?

o 1.2 What Is Data Mining?
o 1.3 What Kinds of Data Can Be Mined?
o 1.4 What Kinds of Patterns Can Be Mined?
o 1.5 What Kinds of Technologies Are Used?

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 2

1.1 Why Data Mining?
 “We are living in the information age” is a popular saying;
however, we are actually living in the data age.
 Terabytes or petabytes of data pour into our computer
networks, the World Wide Web (WWW), and various data
storage devices every day from business, society, science
and engineering, medicine, and almost every other aspect
of daily life.
 This explosive growth of available data volume is a result
of the computerization of our society and the fast
development of powerful data collection and storage tools.
 Businesses worldwide generate gigantic data sets,
including sales transactions, stock trading records, product
descriptions, sales promotions, company profiles and
performance, and customer feedback etc. For example,
large stores, such as Wal-Mart, handle hundreds of
millions of transactions per week at thousands of branches
around the world.
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 3
1.1 Why Data Mining?
 Scientific and engineering practices generate high orders
of petabytes of data in a continuous manner, from remote
sensing, process measuring, scientific experiments,
system performance, engineering observations, and
environment surveillance.
 Global backbone telecommunication networks carry tens of
petabytes of data traffic every day.
 The medical and health industry generates tremendous
amounts of data from medical records, patient monitoring,
and medical imaging.
 Billions of Web searches supported by search engines
process tens of petabytes of data daily.
 Communities and social media have become increasingly
important data sources, producing digital pictures and
videos, blogs, Web communities, and various kinds of
social networks.
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 4
1.1 Why Data Mining?
 This endless sources of explosively growing, widely
available, and gigantic body of data makes our time truly
the data age.
 So, “We are drowning in data, but starving for knowledge!”
 In summary, the abundance of data, coupled with the need
for powerful data analysis tools, has been described as a
data rich but information poor situation.
 The fast-growing, tremendous amount of data, collected
and stored in large and numerous data repositories, has far
exceeded our human ability for comprehension without
powerful tools.
 As a result, data collected in large data repositories
become “data tombs”—data archives that are seldom
visited.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 5

1.1 Why Data Mining?
 Consequently, important decisions are often made based
not on the information-rich data stored in data repositories
but rather on a decision maker’s intuition, simply because
the decision maker does not have the tools to extract the
valuable knowledge embedded in the vast amounts of
data.
 Efforts have been made to develop expert system and
knowledge-based technologies, which typically rely on
users or domain experts to manually input knowledge into
knowledge bases.
 Unfortunately, however, the manual knowledge input
procedure is prone to biases and errors and is extremely
costly and time consuming.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 6

1.1 Why Data Mining?
 The widening gap between data and information calls for
the systematic development of data mining tools that can
turn “data tombs” into “golden nuggets” of knowledge.
 “Necessity is the mother of invention”. Thus, Powerful and
versatile tools are badly needed to automatically uncover
valuable information from the tremendous amounts of
data and to transform such data into organized
knowledge.
 This necessity has led to the birth of data mining. The field
is young, dynamic, and promising.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 7

1.2 What is Data Mining?
 Data mining is the process of discovering or extracting
interesting (non-trivial, implicit, previously unknown and
potentially useful) patterns and knowledge from
huge/large/massive amounts of data.
 The data sources can include databases, data warehouses,
the Web, other information repositories, or data that are
streamed into the system dynamically.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 8

1.2 Data Mining – a misnomer?
 To refer to the mining of gold from rocks or sand, we say
gold mining instead of rock or sand mining. Analogously,
data mining should have been more appropriately named
“knowledge mining from data,” which is unfortunately
somewhat long.
 However, the shorter term, knowledge mining may not
reflect the emphasis on mining from large amounts of data.
 Nevertheless, mining is a vivid term characterizing the
process that finds a small set of precious nuggets from a
great deal of raw material. Thus, such a misnomer carrying
both “data” and “mining” became a popular choice.
 Alternative names: Knowledge discovery (mining) in
databases (KDD), knowledge extraction, data/pattern
analysis, data archaeology, data dredging, information
harvesting, business intelligence etc.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 9

1.2 KDD Process and Data Mining
 Many people treat data mining as a synonym for another
popularly used term, knowledge discovery in database,
or KDD, while others view data mining as merely an
essential step in the process of knowledge discovery.
 The knowledge discovery process includes following steps:
 1. Data cleaning (to remove noise and inconsistent data)
 2. Data integration (where multiple data sources may be
combined)
 3. Data selection (where data relevant to the analysis
task are retrieved from the database)
 4. Data transformation (where data are transformed and
consolidated into forms appropriate for mining by
performing summary or aggregation operations)
 5. Data mining (an essential process where intelligent
methods are applied to extract data patterns)

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 10

1.2 KDD Process and Data Mining
 6. Pattern evaluation (to identify the truly interesting
patterns representing knowledge based on interestingness
measures)
 7. Knowledge presentation (where visualization and
knowledge representation techniques are used to present
mined knowledge to users)
 Steps 1 through 4 are different forms of data
preprocessing, where data are prepared for mining.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 11

1.2 KDD Process

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 12

 Knowledge Discovery (KDD) Process
 This is a view from typical
database systems and data
warehousing communities
Pattern Evaluation

Data Mining

Task-relevant Data

Data Warehouse Selection

Data Cleaning

Data Integration

13
Databases
1.3 What Kinds of Data Can Be Mined?
 Data mining can be applied to any kind of data as long as
the data are meaningful for a target application.
 The most basic forms of data for mining applications are:
 Database data,
 Data warehouse data, and
 Transactional data
 Data mining can also be applied to other forms of data:
 Data streams,
 Ordered/sequence data,
 Graph or networked data,
 Spatial data,
 Text data,
 Multimedia data, and
 The WWW

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 14

1.3.1 Database Data
 A database system, also called a database management
system (DBMS), consists of a collection of interrelated
data, known as a database, and a set of software
programs to manage and access the data.
 A relational database is a collection of tables, each of
which is assigned a unique name. Each table consists of a
set of attributes (columns or fields) and usually stores a
large set of tuples (records or rows).

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 15

1.3.1 Database Data
 Relational data can be accessed by database queries
written in a relational query language (e.g., SQL) or with
the assistance of graphical user interfaces.
 When mining relational databases, we can go further
by searching for trends or data patterns. For example,
data mining systems can analyze customer data to predict
the credit risk of new customers based on their income,
age, education and previous credit information.
 Data mining systems may also detect deviations—that is,
items with sales that are far from those expected in
comparison with the previous year. Such deviations can
then be further investigated. For example, data mining
may discover that there has been a change in packaging
of an item or a significant increase in price.
 Relational databases are one of the most commonly
available and richest information repositories, and thus
they are a major data form in the study of data mining.
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 16
1.3.2 Data Warehouse
 A data warehouse is a repository of information collected
from multiple sources, stored under a unified schema, and
usually residing at a single site.
 Data warehouses are constructed via a process of data
cleaning, data integration, data transformation, data
loading, and periodic data refreshing (ETL process).
 To facilitate decision making, the data in a data warehouse
are organized around major subjects (e.g., customer,
item, supplier, and activity). The data are stored to
provide information from a historical perspective, such as
in the past 6 to 12 months, and are typically summarized.
 For example, rather than storing the details of each sales
transaction, the data warehouse may store a summary of
the transactions per item type for each store or,
summarized to a higher level, for each sales region.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 17

1.3.2 Data Warehouse

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 18

1.3.2 Data Warehouse
 A data warehouse is usually modeled by a multi-
dimensional data structure, called a data cube, in which
each dimension corresponds to an attribute or a set of
attributes in the schema, and each cell stores the value of
some aggregate measure such as count or sum (sales
amount).
 A data cube provides a multidimensional view of data and
allows the pre-computation and fast access of summarized
data
 Online analytical processing (OLAP) operations are used in
multidimensional data which accommodate different user
viewpoints.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 19

1.3.2 Data Warehouse

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 20

1.3.2 Data Warehouse

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 21

1.3.2 Data Warehouse
 OLAP operations include drill-down and roll-up allow the
user to view the data at differing degrees of summarization.
 For instance, we can drill down on sales data summarized
by quarter to see data summarized by month. Similarly,
we can roll up on sales data summarized by city to view
data summarized by country.
 Although data warehouse tools help support data analysis,
additional tools for data mining are often needed for in-
depth analysis. Multidimensional data mining (also
called exploratory multidimensional data mining)
performs data mining in multidimensional space in an OLAP
style.
 That is, it allows the exploration of multiple combinations of
dimensions at varying levels of granularity in data mining,
and thus has greater potential for discovering interesting
patterns representing knowledge.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 22

1.3.3 Transactional Data
 In general, each record in a transactional database
captures a transaction, such as a customer’s purchase, a
flight booking, or a user’s clicks on a web page.
 A transaction typically includes a unique transaction
identity number (trans ID) and a list of the items making
up the transaction, such as the items purchased in the
transaction.
 One might ask,"Which items sold well together?” This kind
of market basket data analysis would enable you to bundle
groups of items together as a strategy for boosting sales.
 For example, given the knowledge that printers are
commonly purchased together with computers, you could
offer certain printers at a steep discount (or even for free)
to customers buying selected computers, in the hopes of
selling more computers (which are often more expensive
than printers).

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 23

1.3.3 Transactional Data
 A traditional database system is not able to perform
market basket data analysis. Fortunately, data mining on
transactional data can do so by mining frequent itemsets,
that is, sets of items that are frequently sold together.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 24

1.3.4 Other Kinds of Data
 Besides relational database data, data warehouse data, and
transaction data, there are many other kinds of data that
have versatile forms and structures.
 Such kinds of data can be seen in many applications:
 Time-related or sequence data (e.g., historical records, stock
exchange data, and time-series and biological sequence data),
 Data streams (e.g., video surveillance and sensor data, which
are continuously transmitted),
 Spatial data (e.g., maps),
 Engineering design data (e.g., the design of buildings, system
components, or integrated circuits),
 Hypertext and multimedia data (including text, image, video,
and audio data),
 Graph and networked data (e.g., social and information
networks), and
 The Web (a huge, widely distributed information repository
made available by the Internet).
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 25
1.3.4 Other Kinds of Data
 These applications bring about new challenges, like how to
handle data carrying special structures (e.g., sequences,
trees, graphs, and networks) and specific semantics (such
as ordering, image, audio and video contents, and
connectivity), and how to mine patterns that carry rich
structures and semantics.
 Various kinds of knowledge can be mined from these kinds
of data.
 Regarding temporal data, for instance, we can mine
banking data for changing trends, which may aid in the
scheduling of bank tellers according to the volume of
customer traffic.
 Stock exchange data can be mined to uncover trends
that could help you plan investment strategies (e.g., the
best time to purchase a stock).

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 26

1.3.4 Other Kinds of Data
 Computer network data streams could be mined to
detect intrusions based on the anomaly of message flows,
which may be discovered by clustering, dynamic
construction of stream models or by comparing the current
frequent patterns with those at a previous time.
 With spatial data, we may look for patterns that describe
changes in metropolitan poverty rates based on city
distances from major highways.
 By mining text data, such as literature on data mining
from the past ten years, we can identify the evolution of
hot topics in the field.
 By mining user comments on products (which are often
submitted as short text messages), we can assess
customer sentiments and understand how well a product is
embraced by a market.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 27

1.3.4 Other Kinds of Data
 From multimedia data, we can mine images to identify
objects and classify them by assigning semantic labels or
tags.
 Web mining can help us learn about the distribution of
information on the WWW in general, characterize and
classify web pages, and uncover web dynamics and the
association and other relationships among different web
pages, users, communities, and web-based activities.
 It is important to keep in mind that, in many applications,
multiple types of data are present. For example, in web
mining, there often exist text data and multimedia data
(e.g., pictures and videos) on web pages, graph data like
web graphs, and map data on some web sites.
 In bioinformatics, genomic sequences, biological networks,
and 3-D spatial structures of genomes may coexist for
certain biological objects.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 28

1.4 What Kinds of Patterns can be Mined?
 Data mining functionalities are used to specify the kinds of
patterns to be found in data mining tasks. There are a
number of data mining functionalities. These include:
 Characterization and discrimination
 The mining of frequent patterns, associations, and
Correlations
 Classification and regression
 Clustering analysis and
 Outlier analysis

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 29

1.4.1 Characterization and Discrimination
 Data characterization is a summarization of the general
characteristics or features of a target class of data. The
data corresponding to the user-specified class are typically
collected by a query.
 For example, to study the characteristics of software
products with sales that increased by 10% in the previous
year, the data related to such products can be collected by
executing an SQL query on the sales database.
 There are several methods for effective data
summarization and characterization. Simple data
summaries based on statistical measures and plots.
 The data cube-based OLAP roll-up operation can be used to
perform user-controlled data summarization along a
specified dimension.
 The output of data characterization can be presented in
various forms. Examples include pie charts, bar charts,
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 30
1.4.1 Characterization and Discrimination
curves, multidimensional data cubes, and multidimensional
tables, including crosstabs.
 Data discrimination is a comparison of the general
features of the target class data objects against the general
features of objects from one or multiple contrasting
classes.
 The target and contrasting classes can be specified by a
user, and the corresponding data objects can be retrieved
through database queries.
 For example, a user may want to compare the general
features of software products with sales that increased by
10% last year against those with sales that decreased by
at least 30% during the same period.
 The methods used for data discrimination are similar to
those used for data characterization.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 31

1.4.2 Mining Frequent Patterns, Associations, and Correlations

 Frequent patterns, as the name suggests, are patterns

that occur frequently in data.
 There are many kinds of frequent patterns, including
 Frequent itemsets,
 Frequent subsequences (known as sequential patterns), and
 Frequent substructures.
 A frequent itemset typically refers to a set of items that
often appear together in a transactional data set—for
example, milk and bread, which are frequently bought
together in grocery stores by many customers.
 A frequently occurring subsequence, such as the pattern
that customers, tend to purchase first a laptop, followed by
a digital camera, and then a memory card, is a (frequent)
sequential pattern.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 32

1.4.2 Mining Frequent Patterns, Associations, and Correlations

 A substructure can refer to different structural forms (e.g.,

graphs, trees, or lattices) that may be combined with
itemsets or subsequences. If a substructure occurs
frequently, it is called a (frequent) structured pattern.
 Mining frequent patterns leads to the discovery of
interesting associations and correlations within data.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 33

1.4.2 Association Analysis
 Suppose that, you to know which items are frequently
purchased together (i.e., within the same transaction).
 An example of such a rule, mined from a transactional
database, is

where X is a variable representing a customer. A

confidence, or certainty, of 50% means that if a customer
buys a computer, there is a 50% chance that she will buy
software as well.
 A 1% support means that 1% of all the transactions under
analysis show that computer and software are purchased
together.
 This association rule involves a single attribute or predicate
(i.e., buys) that repeats. Association rules that contain a
single predicate are referred to as single-dimensional
association rules.
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 34
1.4.2 Association Analysis
 Dropping the predicate notation, the rule can be written
simply as

 Suppose, instead, that we are given the relational database

related to purchases. A data mining system may find
association rules like

 The rule indicates that of the all customers under study,

2% are 20 to 29 years old with an income of $40,000 to
$49,000 and have purchased a laptop (computer).
 There is a 60% probability that a customer in this age and
income group will purchase a laptop.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 35

1.4.2 Association Analysis
 Note that this is an association involving more than one
attribute or predicate (i.e., age, income, and buys).
Adopting the terminology used in multidimensional
databases, where each attribute is referred to as a
dimension, the above rule can be referred to as a
multidimensional association rule.
 Typically, association rules are discarded as uninteresting if
they do not satisfy both a minimum support threshold
and a minimum confidence threshold.
 Additional analysis can be performed to uncover interesting
statistical correlations between associated attribute–value
pairs.
 Frequent itemset mining is a fundamental form of frequent
pattern mining.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 36

Association Analysis : Support & Conf.
 Let I = { i1,i2,…,im } be a set of items
 Let D = { t1,t2…,tn } be a set of transactions where every
ti is a set of items such that ti Є I
 Each transaction is associated with an id TID

 An association rule is of the form

A B, where A Є I, B Є I and A ∩ B = Ø
 A rule A B, has support s means
- the percentage of transactions in D that contains A U B
s=P(AUB)
 A rule A B, has confidence c means
- the percentage of transaction in D containing A that also
contains B,
c = P ( B | A ) = P(A U B) / P(A)

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 37

Association Analysis : Support & Conf.

 Support of A C is
s = n(AUC)/n(D) TID Itemset
s = 2/4 X 100% = 50%
100 A,B,C
 Confidence of A C is
c = n(AUC) / n(A) 200 A,C
c = 2/3 X 100% = 66.6%

 Confidence of C A is
300 A,D
c = n(AUC) / n(C)
c = 2/2 X 100% = 100% 400 B,E,F

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 38

1.4.3 Classification and Regression for Predictive Analysis
 Classification is the process of finding a model (or
function) that describes and distinguishes data classes or
concepts.
 The model are derived based on the analysis of a set of
training data (i.e., data objects for which the class labels
are known). The model is used to predict the class label of
objects for which the class label is unknown.
 The derived model may be represented in various forms:
 classification rules (i.e., IF-THEN rules),
 decision trees,
 mathematical formulae, or
 neural networks
 A decision tree is a flowchart-like tree structure, where
each node denotes a test on an attribute value, each
branch represents an outcome of the test, and tree leaves
represent classes or class distributions.
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 39
1.4.3 Classification and Regression for Predictive Analysis

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 40

1.4.3 Classification and Regression for Predictive Analysis

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 41

1.4.3 Classification and Regression for Predictive Analysis

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 42

1.4.3 Classification and Regression for Predictive Analysis
 Decision trees can easily be converted to classification
rules.
 A neural network, when used for classification, is
typically a collection of neuron-like processing units with
weighted connections between the units.
 There are many other methods for constructing
classification models, such as naïve Bayesian classification,
support vector machines (SVM), and k-nearest-neighbor
classification.
 Whereas classification predicts categorical (discrete,
unordered) labels, regression models continuous-valued
functions.
 That is, regression is used to predict missing or unavailable
numerical data values rather than (discrete) class labels.
The term prediction refers to both numeric prediction and
class label prediction.
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 43
1.4.3 Classification and Regression for Predictive Analysis
 Regression analysis is a statistical methodology that is
most often used for numeric prediction, although other
methods exist as well.
 Regression also encompasses the identification of
distribution trends based on the available data.
 Classification and regression may need to be preceded by
relevance analysis, which attempts to identify attributes
that are significantly relevant to the classification and
regression process. Such attributes will be selected for the
classification and regression process. Other attributes,
which are irrelevant, can then be excluded from
consideration.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 44

1.4.4 Cluster Analysis
 Unlike classification and regression, which analyze class-
labeled (training) data sets, clustering analyzes data
objects without consulting class labels.
 In many cases, class labeled data may simply not exist at
the beginning. Clustering can be used to generate class
labels for a group of data.
 The objects are clustered or grouped based on the principle
of maximizing the intraclass similarity and minimizing the
interclass similarity.
 That is, clusters of objects are formed so that objects
within a cluster have high similarity in comparison to one
another, but are rather dissimilar to objects in other
clusters.
 Each cluster so formed can be viewed as a class of objects,
from which rules can be derived. Clustering can also
facilitate taxonomy formation, that is, the organization
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 45
1.4.4 Cluster Analysis
of observations into a hierarchy of classes that group
similar events together.
 Example of Cluster analysis: Cluster analysis can be
performed on customer data to identify homogeneous
subpopulations of customers. These clusters may represent
individual target groups for marketing.
 Figure 1.10 shows a 2-D plot of customers with respect to
customer locations in a city. Three clusters of data points
are evident.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 46

1.4.4 Cluster Analysis

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 47

1.4.5 Outlier Analysis
 A data set may contain objects that do not comply with the
general behavior or model of the data. These data objects
are outliers.
 Many data mining methods discard outliers as noise or
exceptions. However, in some applications (e.g., fraud
detection) the rare events can be more interesting than the
more regularly occurring ones. The analysis of outlier data
is referred to as outlier analysis or anomaly mining.
 Outliers may be detected using statistical tests that
assume a distribution or probability model for the data, or
using distance measures where objects that are remote
from any other cluster are considered outliers.
 Outlier analysis may uncover fraudulent usage of credit
cards by detecting purchases of unusually large amounts
for a given account number in comparison to regular
charges incurred by the same account. Outlier values may
also be detected with respect to the locations and types of
purchase, or the purchase
15-Jan-19 frequency.
Abu Ahmed Ferdaus, CSE, DU. 48
1.4.6 Are All Patterns Interesting?
 A data mining system has the potential to generate
thousands or even millions of patterns, or rules.
 “Are all of the patterns interesting?”
 Typically, the answer is no—only a small fraction of the
patterns potentially generated would actually be of interest
to a given user.
 This raises some serious questions for data mining:
 1. What makes a pattern interesting?
 2. Can a data mining system generate all of the interesting
patterns? Or,
 3. Can the system generate only the interesting ones?
 A pattern is interesting if it is
 (1) easily understood by humans,
 (2) valid on new or test data with some degree of certainty,
 (3) potentially useful, and
 (4) novel.
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 49
1.4.6 Are All Patterns Interesting?
 A pattern is also interesting if it validates a hypothesis that
the user sought (required) to confirm. An interesting
pattern represents knowledge.
 Interestingness measures are: 1. Objective and 2.
Subjective
 Several objective measures of pattern interestingness
exist. These are based on the structure of discovered
patterns and the statistics underlying them.
 An objective measure for association rules of the form X 
Y is rule support, representing the percentage of
transactions from a transaction database that the given
rule satisfies.
 This is taken to be the probability P(X  Y), where X  Y
indicates that a transaction contains both X and Y, that is,
the union of itemsets X and Y.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 50

1.4.6 Are All Patterns Interesting?
 Another objective measure for association rules is
confidence, which assesses the degree of certainty of the
detected association.
 This is taken to be the conditional probability P(Y | X), that
is, the probability that a transaction containing X also
contains Y.
 More formally, support and confidence are defined as

 In general, each interestingness measure is associated with

a threshold, which may be controlled by the user.
 For example, rules that do not satisfy a confidence
threshold of, say, 50% can be considered uninteresting.
Rules below the threshold likely reflect noise, exceptions,
or minority cases and are probably of less value.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 51

1.4.6 Are All Patterns Interesting?
 Other objective interestingness measures include accuracy
and coverage for classification (IF-THEN) rules.
 In general terms, accuracy tells us the percentage of data
that are correctly classified by a rule. Coverage is similar to
support, in that it tells us the percentage of data to which a
rule applies.
 Although objective measures help identify interesting
patterns, they are often insufficient unless combined with
subjective measures that reflect a particular user’s needs
and interests.
 For example, patterns describing the characteristics of
customers who shop frequently should be interesting to the
marketing manager, but may be of little interest to other
analysts studying the same database for patterns on
employee performance.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 52

1.4.6 Subjective Interestingness
 Furthermore, many patterns that are interesting by
objective standards may represent common sense and,
therefore, are actually uninteresting.
 Subjective interestingness measures are based on user
beliefs in the data. These measures find patterns
interesting if the patterns are unexpected (contradicting a
user’s belief) or offer strategic information on which the
user can act.
 In the latter case, such patterns are referred to as
actionable. For example, patterns like “a large earthquake
often follows a cluster of small quakes” may be highly
actionable if users can act on the information to save lives.
 Patterns that are expected can be interesting if they
confirm a hypothesis that the user wishes to validate or
they resemble a user’s hunch (guess or intuition).

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 53

1.4.6 Are All Patterns Interesting?
 The second question—“Can a data mining system generate
all of the interesting patterns?”
 It refers to the completeness of a data mining algorithm.
It is often unrealistic and inefficient for data mining
systems to generate all possible patterns.
 Instead, user provided constraints and interestingness
measures should be used to focus the search.
 For some mining tasks, such as association, this is often
sufficient to ensure the completeness of the algorithm.
 Association rule mining is an example where the use of
constraints and interestingness measures can ensure the
completeness of mining.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 54

1.4.6 Are All Patterns Interesting?
 Finally, the third question—“Can a data mining system
generate only interesting patterns?”—
 It is an optimization problem in data mining. It is highly
desirable for data mining systems to generate only
interesting patterns.
 This would be efficient for users and data mining systems
because neither would have to search through the patterns
generated to identify the truly interesting ones.
 Progress has been made in this direction; however, such
optimization remains a challenging issue in data mining.
 Measures of pattern interestingness are essential for
filtering out the uninteresting ones.
 More important, such measures can be used to guide and
constrain the discovery process, improving the search
efficiency by pruning away subsets of the pattern space
that do not satisfy pre-specified interestingness constraints.
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 55
1.5 Which Technologies Are Used?
 As a highly application-driven domain, data mining has
incorporated many techniques from other domains such as
 Statistics,
 Machine learning,
 Pattern recognition,
 Database and data warehouse systems,
 Information retrieval,
 Visualization,
 Algorithms,
 High-performance computing etc.
 The interdisciplinary nature of data mining research and
development contributes significantly to the success of
data mining and its extensive applications.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 56

1.5 Which Technologies Are Used?

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 57

 1.5.1 Statistics
 Statistics studies the collection, analysis, interpretation or
explanation, and presentation of data. Data mining has an
inherent connection with statistics.
 A statistical model is a set of mathematical functions that
describe the behavior of the objects in a target class in
terms of random variables and their associated probability
distributions. Statistical models are widely used to model
data and data classes.
 For example, in data mining tasks like data characterization
and classification, statistical models of target classes can be
built. In other words, such statistical models can be the
outcome of a data mining task. Alternatively, data mining
tasks can be built on top of statistical models.
 For example, we can use statistics to model noise and
missing data values. Then, when mining patterns in a large
data set, the data mining process can use the model to help
identify and handle noisy or missing values in the data.
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 58
 1.5.2 Machine Learning
 Machine learning investigates how computers can learn
(or improve their performance) based on data.
 A main research area is for computer programs to
automatically learn to recognize complex patterns and
make intelligent decisions based on data.
 For example, a typical machine learning problem is to
program a computer so that it can automatically recognize
handwritten postal codes on mail after learning from a set
of examples.
 Machine learning is a fast-growing discipline. The learning
process can be:
 Supervised learning
 Unsupervised learning
 Semi-supervised learning
 Active learning

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 59

 1.5.2 Machine Learning
 Supervised learning is basically a synonym for
classification. The supervision in the learning comes from
the labeled examples in the training data set.
 For example, in the postal code recognition problem, a set
of handwritten postal code images and their corresponding
machine-readable translations are used as the training
examples, which supervise the learning of the classification
model.
 Unsupervised learning is essentially a synonym for
clustering. The learning process is unsupervised since the
input examples are not class labeled. Typically, we may use
clustering to discover classes within the data.
 For example, an unsupervised learning method can take, as
input, a set of images of handwritten digits. Suppose that it
finds 10 clusters of data. These clusters may correspond to
the 10 distinct digits of 0 to 9, respectively.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 60

 1.5.2 Machine Learning
 However, since the training data are not labeled, the
learned model cannot tell us the semantic meaning of the
clusters found.
 Semi-supervised learning is a class of machine learning
techniques that make use of both labeled and unlabeled
examples when learning a model.
 In one approach, labeled examples are used to learn class
models and unlabeled examples are used to refine the
boundaries between classes.
 For a two-class problem, we can think of the set of
examples belonging to one class as the positive examples
and those belonging to the other class as the negative
examples.
 In Figure 1.12, if we do not consider the unlabeled
examples, the dashed line is the decision boundary that
best partitions the positive examples from the negative
examples.
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 61
 1.5.2 Machine Learning
 Using the unlabeled examples, we can refine the decision
boundary to the solid line. Moreover, we can detect that the
two positive examples at the top right corner, though
labeled, are likely noise or outliers.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 62

 1.5.2 Machine Learning
 Active learning is a machine learning approach that lets
users play an active role in the learning process.
 An active learning approach can ask a user (e.g., a domain
expert) to label an example, which may be from a set of
unlabeled examples or synthesized by the learning program.
 The goal is to optimize the model quality by actively
acquiring knowledge from human users, given a constraint
on how many examples they can be asked to label.
 There are many similarities between data mining and
machine learning. For classification and clustering tasks,
machine learning research often focuses on the accuracy of
the model.
 In addition to accuracy, data mining research places strong
emphasis on the efficiency and scalability of mining methods
on large data sets, as well as on ways to handle complex
types of data and explore new, alternative methods.
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 63
 1.5.3 Database Systems and Data Warehouses
 Database systems research focuses on the creation,
maintenance, and use of databases for organizations and
end-users.
 Particularly, database systems researchers have established
highly recognized principles in data models, query
languages, query processing and optimization methods,
data storage, and indexing and accessing methods.
 Database systems are often well known for their high
scalability in processing very large, relatively structured data
sets.
 Many data mining tasks need to handle large data sets or
even real-time, fast streaming data.
 Therefore, data mining can make good use of scalable
database technologies to achieve high efficiency and
scalability on large data sets. Moreover, data mining tasks
can be used to extend the capability of existing database
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 64
 1.5.3 Database Systems and Data Warehouses
systems to satisfy advanced users’ sophisticated data
analysis requirements.
 Recent database systems have built systematic data
analysis capabilities on database data using data
warehousing and data mining facilities.
 A data warehouse integrates data originating from
multiple sources and various timeframes. It consolidates
data in multidimensional space to form partially materialized
data cubes.
 The data cube model not only facilitates OLAP in multi-
dimensional databases but also promotes multidimensional
data mining.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 65

 1.5.4 Information Retrieval
 Information retrieval (IR) is the science of searching for
documents or information in documents.
 Documents can be text or multimedia, and may reside on
the Web. The differences between traditional information
retrieval and database systems are twofold:
 Information retrieval assumes that (1) the data under
search are unstructured; and (2) the queries are formed
mainly by keywords, which do not have complex structures.
 Increasingly large amounts of text and multimedia data
have been accumulated and made available online due to
the fast growth of the Web and applications such as digital
libraries, digital governments, and health care information
systems.
 Their effective search and analysis have raised many
challenging issues in data mining. Therefore, text mining
and multimedia data mining, integrated with information
retrieval methods, have become increasingly important.
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 66
 1.8 Summary
 Data mining is the process of discovering interesting
patterns from massive amounts of data. As a knowledge
discovery process, it typically involves data cleaning, data
integration, data selection, data transformation, pattern
discovery, pattern evaluation, and knowledge presentation.
 A pattern is interesting if it is valid on test data with some
degree of certainty, novel, potentially useful (e.g., can be
acted on or validates a hunch about which the user was
curious), and easily understood by humans. Interesting
patterns represent knowledge. Measures of pattern
interestingness, either objective or subjective, can be
used to guide the discovery process.
 Data mining can be conducted on any kind of data as long
as the data are meaningful for a target application, such as
database data, data warehouse data, transactional data,
and advanced data types. Advanced data types include
time-related or sequence data, data streams, spatial and
15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 67
 1.8 Summary
spatiotemporal data, text and multimedia data, graph and
networked data, and Web data.
 A data warehouse is a repository for long-term storage of
data from multiple sources, organized so as to facilitate
management decision making. The data are stored under a
unified schema and are typically summarized. Data
warehouse systems provide multidimensional data analysis
capabilities, collectively referred to as online analytical
processing (OLAP).
 Multidimensional data mining (also called exploratory
multidimensional data mining) integrates core data
mining techniques with OLAP-based multidimensional
analysis. It searches for interesting patterns among
multiple combinations of dimensions (attributes) at varying
levels of abstraction, thereby exploring multidimensional
data space.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 68

 1.8 Summary
 Data mining functionalities are used to specify the kinds
of patterns or knowledge to be found in data mining tasks.
The functionalities include characterization and discrimi-
nation; the mining of frequent patterns, associations, and
correlations; classification and regression; cluster analysis;
and outlier detection. As new types of data, new
applications, and new analysis demands continue to
emerge, there is no doubt we will see more and more novel
data mining tasks in the future (EDM).
 Data mining, as a highly application-driven domain, has
incorporated technologies from many other domains.
These include statistics, machine learning, database and
data warehouse systems, and information retrieval. The
interdisciplinary nature of data mining research and
development contributes significantly to the success of
data mining and its extensive applications.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 69

 1.8 Summary
 Data mining has many successful applications, such as
business intelligence, Web search, bioinformatics, health
informatics, finance, digital libraries, education and digital
governments.
 There are many challenging issues in data mining
research. Areas include mining methodology, user
interaction, efficiency and scalability, and dealing with
diverse data types. Data mining research has strongly
impacted society and will continue to do so in the future.

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 70

 Thank You

15-Jan-19 Abu Ahmed Ferdaus, CSE, DU. 71