DATA WAREHOUSING AND DATA MINING

(5M EACH)
1. Illustrate the alternative methods for generating frequent item
sets with suitable examples.
Ans:
Alternative Methods for Generating Frequent Itemsets

The generation of frequent itemsets is a fundamental problem in association rule mining. Beyond the
Apriori algorithm, several alternative methods exist to address its limitations, including handling large
and dense datasets more efficiently. Below, we illustrate these methods with suitable examples.

1. Traversal of Itemset Lattice

Frequent itemsets can be discovered by traversing the lattice structure of itemsets. The traversal
method significantly affects the algorithm's performance.

a) General-to-Specific Search (Breadth-First Search)

 Approach: Start with smaller itemsets (e.g., frequent 1-itemsets) and progressively generate
larger itemsets.

 Example:

o Transactions: {a, b, c}, {a, b}, {b, c, d}, {a, c, d}

o Frequent 1-itemsets: {a}, {b}, {c}, {d}

o Generate candidates for 2-itemsets: {a, b}, {b, c}, {a, c}, {c, d}

o Evaluate candidates to find frequent 2-itemsets.

b) Specific-to-General Search (Depth-First Search)

 Approach: Begin with more specific itemsets (e.g., maximal frequent itemsets) and work
backward to subsets.

 Example:

o Transactions: {a, b, c, d}, {a, b, c}, {b, c}, {a, d}

o Start with itemset {a, b, c, d}.

o If frequent, prune subsets like {a, b, c} and {a, b} since they are guaranteed frequent.

c) Bidirectional Search
  Approach: Combine general-to-specific and specific-to-general strategies.

 Example: Start with {a, b} and {b, c, d}. Evaluate both subsets and supersets simultaneously
to locate frequent itemsets faster.

2. Equivalence Classes

This method groups itemsets into equivalence classes based on shared characteristics.

a) Prefix-Based Classes

 Approach: Partition itemsets by common prefixes.

 Example:

o Transactions: {a, b, c}, {a, b}, {b, c}

o Prefix a: Itemsets {a}, {a, b}, {a, c}

o Prefix b: Itemsets {b}, {b, c}

o Each prefix group is processed independently.

b) Suffix-Based Classes

 Approach: Partition itemsets by common suffixes.

 Example: For the same transactions, itemsets {a, b}, {b, c} share suffix b.

3. Breadth-First vs. Depth-First Traversal

a) Breadth-First Search

 Example:

o Start with frequent 1-itemsets {a}, {b}, {c}.

o Expand to {a, b}, {a, c}, {b, c} and so on.

b) Depth-First Search

 Example:

o Start with {a} and expand to {a, b}, {a, b, c} until an infrequent itemset is reached.

o Backtrack to explore other branches like {b, c, d}.

4. Representation of Transaction Data

The transaction data format can significantly affect performance.

a) Horizontal Data Layout

 Approach: Each transaction is represented as a list of items.

 Example:

o Transactions: {1: [a, b, c]}, {2: [a, b]}, {3: [b, c, d]}

o Use intersection operations to find frequent itemsets.

b) Vertical Data Layout

 Approach: Store a list of transactions (TID) for each item.

 Example:

o {a: [1, 2, 3]}, {b: [1, 2, 3]}, {c: [1, 3]}

o Compute support for {a, b} by intersecting TID lists: {1, 2, 3} ∩ {1, 2, 3} = {1, 2, 3}.

Key Takeaways

1. General-to-Specific works well when itemsets are not too long.

2. Specific-to-General is more efficient for dense datasets with maximal frequent itemsets.

3. Bidirectional Search combines both strategies for faster convergence.

4. Equivalence Classes reduce search space by partitioning itemsets.

5. Data Representation (horizontal vs. vertical) impacts I/O performance and memory usage.

Each method has strengths and is suitable for different transaction configurations. Selecting the
appropriate method is crucial for optimizing frequent itemset mining.

2. A healthcare organization has implemented a machine learning

system to classify patient records into different risk categories for
diabetes (Low, Medium, High). The system uses a rule-based
classifier where rules are defined based on patient attributes such
as Age, BMI (Body Mass Index), Blood Sugar Level, and Family
History of diabetes. Develop Classifier rules for the above
scenario and Explain How Sequential Covering algorithm in rule-
based classifiers works for above scenario.
3. Outline the drawbacks of Apriori Algorithm with relevant
examples.
Ans:
Drawbacks of the Apriori Algorithm

The Apriori algorithm is widely used for mining frequent itemsets and generating
association rules in transaction datasets. However, it has notable limitations that affect its
performance and scalability.

1. High Computational Cost

 Explanation: Apriori generates a large number of candidate itemsets in each iteration,

even though many of them may not be frequent.

 Example: Consider a dataset with 100 items. For a frequent itemset size of 3, Apriori
must evaluate all combinations of 3 items ((100/3)=161,700). This leads to excessive
computation, especially for large datasets.

2. Multiple Database Scans

 Explanation: The algorithm requires scanning the entire dataset multiple times, once
for each iteration (i.e., for frequent 1-itemsets, 2-itemsets, etc.).

 Example: In a retail dataset with millions of transactions, scanning the dataset

repeatedly for each level of frequent itemsets significantly increases I/O overhead and
execution time.

3. Inefficiency with Dense Datasets

 Explanation: In dense datasets, many items often co-occur in transactions, leading to

a combinatorial explosion of candidate itemsets.

 Example: In a dataset of supermarket transactions where items like bread, milk, and
eggs often co-occur, the number of frequent itemsets becomes very large. This
overwhelms memory and computational resources.

4. Candidate Generation and Storage Overhead

  Explanation: The algorithm generates a vast number of candidate itemsets, which can
cause memory and storage issues.

 Example: In a dataset with 50 items, generating frequent itemsets up to size 5 could

require storing thousands of candidate itemsets in memory, even though many are
eventually pruned.

5. Difficulty Handling Low Support Thresholds

 Explanation: A low minimum support threshold leads to more candidate itemsets

being considered, increasing the computational cost exponentially.

 Example: Setting the minimum support threshold to 0.5% in a dataset with 1 million
transactions may result in generating thousands of infrequent itemsets that are
eventually pruned.

6. Limited Scalability

 Explanation: Apriori struggles with large-scale datasets because of its high

computational and memory requirements.

 Example: Applying Apriori to a dataset with billions of transactions, such as an e-

commerce transaction database, would be infeasible without substantial optimization.

7. Not Suitable for Stream Data

 Explanation: Apriori is designed for static datasets and cannot efficiently handle real-
time or streaming data.

 Example: In a system that processes live user activity on a website, Apriori cannot
dynamically update frequent itemsets as new data arrives.

Alternatives to Overcome Apriori Drawbacks

1. FP-Growth Algorithm: Reduces the need for candidate generation by constructing a

compact tree structure.

2. ECLAT Algorithm: Uses a vertical data format to efficiently compute intersections

of transaction IDs.

3. Parallel and Distributed Approaches: Use frameworks like MapReduce or Spark

for scalability.
4. Construct an FP-Tree for the following transactions with a
minimum support of 2. Then, draw the resulting FP-Tree
structure.
TI Items
D
1 {a, b, d, e}
2 {b, c, e}
3 {a, b, c, e}
4 {b, c, e, d}
5 {a, b, c, e}

5.
TID 1 2 3 4 5 6 7 8

Items a b, a, a, a a, a a,
, ,
c, c, d b b
b , b , ,
d d,
,
e c, c
e
c
d

Generate the frequent items sets for the above data with
support=50%, by making use of Association analysis algorithm
which requires minimal database scans
6. Choose the steps which are required to build Decision tree
using Hunts Algorithm.
Ans:

7. Demonstrate how a Rule-Based Classifier Works with relevant

examples.
Ans:
How a Rule-Based Classifier Works

A rule-based classifier uses if-then rules to classify records. Each rule has a condition
(antecedent) and a class label (consequent). The classifier identifies the rule triggered by a
test record and uses it to assign the class label.

Example Rule Set

Below is an example rule set for classifying vertebrates:

1. r1: If Body Temperature = cold-blooded → Non-mammals

2. r2: If Body Temperature = warm-blooded AND Gives Birth = yes → Mammals

3. r3: If Body Temperature = warm-blooded AND Gives Birth = no → Non-mammals

Steps to Classify Records

1. Classify a Record

 Example: Classify a lemur (warm-blooded, gives birth).

 The lemur satisfies the conditions of rule r2:

Result: Lemur is classified as a Mammal.

2. Handle Conflicts

 Example: Classify a turtle (cold-blooded, semi-aquatic, scales).

 The turtle satisfies rules r1 (Non-mammals) and another rule (e.g., Amphibians).

 Conflict Resolution:

o Use Ordered Rules: Pick the rule with the highest priority.

o Use Unordered Rules: Tally votes from all matching rules (weighted by rule
accuracy if needed).

3. Default Rule

 Example: Classify a dogfish shark (cold-blooded, aquatic, scales), which matches no

rule.

 Apply a default rule: Assign to the majority class in the dataset.

Result: If most animals in the dataset are Non-mammals, classify the shark as a Non-
mammal.

Properties of Rule Sets

1. Mutually Exclusive Rules:

o Each record matches at most one rule.

o Example: If only r1, r2, and r3 exist, a lemur matches only r2.

2. Exhaustive Rules:

o Every record is covered by at least one rule.

o Example: Add a default rule like rd: () → Non-mammals to cover

uncategorized records.
Approaches for Conflict Resolution

1. Ordered Rules (Decision List):

o Rules are ranked by priority (e.g., accuracy).

o Example: If rules r1 and r5 match, the higher-priority rule decides the

classification.

2. Unordered Rules:

o Votes from matching rules are tallied.

o Example: If rules classify as Amphibians (3 votes) or Non-mammals (2 votes),

assign Amphibians.

Summary of Advantages

 Ordered Rules: Simple to classify but sensitive to rule order.

 Unordered Rules: Handles conflicts better but is computationally expensive.

8. You are working on a machine learning project for a

healthcare company to develop a model that can predict whether
a patient has a particular medical condition based on their
symptoms and test results. The dataset contains labeled examples,
and the team decides to start with a simple, instance-based
learning method to quickly classify new patient records.
Your colleague suggests using the Nearest Neighbor classifier for
this task.
How would you describe the Nearest Neighbor classifier in this
context?
What are its key characteristics that make it suitable or
unsuitable for this type of problem?
Ans:
Description of the Nearest Neighbor Classifier in Context

The Nearest Neighbor (NN) classifier is a simple, instance-based learning method that
classifies a new patient record by comparing it to the most similar records in the dataset. It
assumes that records with similar symptoms and test results are likely to belong to the same
class. The classification is based on proximity in a feature space, typically measured using a
distance metric like Euclidean distance.

For example:

 A new patient's record is compared to all existing patient records in the dataset.

 The record is assigned to the class (e.g., "Condition Present" or "Condition Absent")
of the nearest neighbor or a majority class among the k-nearest neighbors.

Key Characteristics of the Nearest Neighbor Classifier

1. Advantages

 No Training Phase: NN does not require a training phase, making it quick to set up
and computationally inexpensive for small datasets.

 Flexible to Complex Relationships: It can model non-linear decision boundaries,

which is useful if the relationship between symptoms/test results and the condition is
complex.

 Interpretable: Easy to explain as it relies on proximity to similar records.

2. Challenges

 Scalability: NN requires comparing the test record to all training records, which can
be computationally expensive for large datasets.

 Sensitive to Noise: Outliers or mislabeled data can adversely affect classification

accuracy.

 Feature Scaling: Features must be normalized (e.g., symptoms and test results on
different scales) to ensure fair distance computation.

 Memory Usage: NN needs to store the entire dataset, which can be impractical for
large datasets.

 Imbalanced Data: If one class dominates the dataset, NN might bias towards that
class unless adjustments (e.g., weighted voting) are made.

Suitability for the Healthcare Context

When It Is Suitable

 Small Dataset: If the dataset is relatively small and representative, NN can perform
well.
  Interpretable Results: NN provides clear reasoning for classification by pointing to
similar records.

 Quick Deployment: If a rapid initial model is needed, NN is easy to implement and

tune.

When It Is Unsuitable

 Large Dataset: Healthcare datasets can be large, making NN computationally

intensive.

 High Dimensionality: If there are many symptoms and test results, the "curse of
dimensionality" can reduce NN’s performance.

 Noisy or Incomplete Data: Healthcare data often contain noise or missing values,
which NN is not inherently robust against.

 Critical Decisions: If the predictions significantly impact patient care, the lack of
robustness or explainability for all cases might make NN less ideal compared to more
advanced models.

Conclusion

The Nearest Neighbor classifier can serve as a quick, baseline model for the healthcare
dataset. However, its scalability, sensitivity to noise, and dependence on feature scaling may
limit its utility as the primary method. It’s advisable to combine NN with preprocessing steps
(e.g., feature selection and normalization) or consider transitioning to more sophisticated
models like decision trees or ensemble methods for better performance and interpretability in
the long term.

9. Examine can we apply Bayes Theorem to a real-world scenario,

such as predicting the likelihood of a disease based on test results.
Ans:
Applying Bayes Theorem in a Real-World Scenario: Predicting the Likelihood of a Disease

Bayes Theorem is a mathematical framework used to update probabilities based on new evidence. In
the context of healthcare, it is highly applicable for predicting the likelihood of a disease based on test
results.

Bayes Theorem Formula

P(A∣B) = [P(B∣A)⋅P(A)] / P(B)

Where:
  P(A∣B) Posterior probability – the probability of having the disease (A) given a positive test
result (B).

 P(B∣A) Likelihood – the probability of the test being positive if the patient has the disease.

 P(A) Prior probability – the prevalence of the disease in the population.

 P(B) Marginal probability – the overall probability of the test being positive.

Example: Predicting a Disease

Suppose a healthcare company wants to predict the likelihood of a patient having a disease DDD
based on a positive test result T+. Here are the given data:

1. Prevalence of the disease (P(D)) 0.01 (1% of the population has the disease).

2. Sensitivity (P(T+∣D)): 0.95 (95% of people with the disease test positive).

3. Specificity (P(T−∣Dc)): 0.90 (90% of people without the disease test negative).

Interpretation

The probability of having the disease given a positive test result is approximately 8.76%, even though
the test has high sensitivity and specificity. This result demonstrates how the low prevalence of the
disease significantly impacts the posterior probability.

Real-World Implications
 1. Importance of Prior Probability: In low-prevalence diseases, even highly accurate tests can
lead to a high number of false positives.

2. Context Matters: Bayes Theorem helps healthcare providers interpret diagnostic test results
in the context of disease prevalence.

3. Decision-Making: By calculating the posterior probability, doctors can decide whether

additional testing or treatment is warranted.

Bayes Theorem is a powerful tool for improving the accuracy of predictions in medical diagnostics
and beyond, illustrating how probabilities can be updated effectively based on new evidence.

10. A real estate agency uses a KNN classifier to predict the type
of property (Residential, Commercial, or Industrial) based on
historical data. The features used for classification are: Size (in
square feet), Number of Floors and Distance from City Centre (in
miles). The agency has labelled data for existing properties, and
the KNN algorithm is configured to use k=3 (i.e., the 3 nearest
neighbors are considered for classification). The property
classification is given the table (a)

Size (sq. Distance

Property ID Floors Type
ft.) (miles)

P1 1500 1 5 Residential

P2 3000 2 3 Commercial

P3 1000 1 7 Residential

P4 4000 3 4 Commercial

P5 8000 1 10 Industrial

Examine how KNN algorithm works for above data set to classify
the property?
11. Apply your understanding of clustering techniques with
respect to the following:
(a) Density-Based Clustering – How does this method identify
clusters based on density, and how would you use it to handle
noise and outliers in a given dataset?
(b) Graph-Based Clustering – Demonstrate how this method
clusters data by representing it as a graph, and explain how the
structure of the graph influences the clustering process.
Ans:

12. Demonstrate the DBSCAN algorithm and explain how it

identifies clusters in a dataset. Provide an example to illustrate
how DBSCAN groups data points based on density and handles
noise.
Ans:

13. Given a set of data points, Interpret how you would use
Agglomerative Hierarchical Clustering to identify clusters,
including the criteria for merging clusters.
Ans:

14. A food delivery company wants to group its customers based

on their ordering behavior. The features considered are: Average
Order Value (in dollars), Frequency of Orders (per month),
Preferred Delivery Time (Morning, Afternoon, Evening).The
company aims to optimize its marketing strategy by targeting
specific clusters, such as frequent low-spenders or occasional
high-spenders. Apply clustering algorithm by considering
features appropriately.
Ans:
15. Apply your knowledge of Agglomerative Hierarchical
Clustering with respect to the different approaches used to
generate clusters. Demonstrate how each approach (such as single
linkage, complete linkage, and average linkage) impacts the
clustering process and the final result.
Ans:
To analyze how different approaches in Agglomerative Hierarchical Clustering (AHC)
(e.g., single linkage, complete linkage, and average linkage) impact the clustering process
and final results, let’s explore each approach with an example dataset.

Dataset

We'll use six two-dimensional points as outlined in the text:

The Euclidean distances between the points are provided in Table 8.4 of the textbook.

Single Linkage (Minimum Distance)

Definition:

The distance between two clusters is defined as the minimum distance between any two
points in the clusters. This results in chaining clusters that may form elongated shapes.

Process:

1. Merge the closest two points first (smallest distance from the matrix: p3 and p6 at
0.11).

2. At each step, merge the two clusters or points with the smallest minimum distance.

3. Repeat until all points form one cluster.

Characteristics:

 Good at handling non-elliptical shapes.

 Sensitive to noise and outliers.

Result:
The dendrogram shows tight groupings of nearby points, with clusters forming based on
minimum distances. For the given dataset:

 p3 and p6 merge first.

 Larger clusters tend to form as long chains, leading to potential clustering errors in the
presence of outliers.

Complete Linkage (Maximum Distance)

Definition:

The distance between two clusters is defined as the maximum distance between any two
points in the clusters. This approach focuses on the largest distance and tends to form more
compact clusters.

Process:

1. Merge the pair of clusters with the smallest maximum distance.

2. Update the distance matrix accordingly, considering the new maximum distances
between clusters.

Characteristics:

 Less sensitive to noise and outliers than single linkage.

 Tends to split large clusters if they contain points far apart.

 Prefers globular shapes.

Result:

Clusters are more compact compared to single linkage. For the given dataset:

 The clustering process ensures tighter groupings by avoiding long chains of points.

Average Linkage (Group Average)

Definition:

The distance between two clusters is defined as the average of all pairwise distances
between points in the two clusters. It balances the extremes of single and complete linkage.

Process:

1. Merge the pair of clusters with the smallest average distance.

 2. Update the distance matrix to reflect the new average distances between clusters.

Characteristics:

 Provides a middle ground between single and complete linkage.

 Handles noise and outliers better than single linkage.

 Forms clusters of moderate compactness and separation.

Result:

For the given dataset:

 The average distance criterion results in clusters that balance proximity and spread.

 Clustering results may differ from single or complete linkage but often produce
intuitive groupings.

Comparison of Results

Method Characteristics Final Clusters (Example)

Single Forms elongated, loose clusters; sensitive to noise p3-p6, p2-p5, then merges
Linkage and outliers. as a chain.

Complete Produces compact, spherical clusters; robust to p3-p6, p3-p6-p4, other

Linkage noise but may split large clusters. points cluster later.

Average Balances the other two methods, forming p3-p6, p3-p6-p4, then
Linkage moderately compact and balanced clusters. merges other points.

Visual Interpretation

 Single Linkage: Dendrograms are long and often show a gradual merging process.

 Complete Linkage: Dendrograms reveal tight groupings that merge late.

 Average Linkage: Dendrograms provide intermediate clustering results with

moderate merging at various levels.

Conclusion

The choice of linkage criterion significantly impacts the clustering process and final clusters.
Single linkage is suited for detecting elongated clusters but struggles with noise. Complete
linkage is ideal for compact clusters but may split larger ones. Average linkage offers a
balance, making it a versatile option for many datasets.
16. Outline DENCLUE algorithm with relevant examples
Ans:
Outline of the DENCLUE Algorithm

DENCLUE (DENsity CLUstEring) is a density-based clustering algorithm that identifies

clusters by modeling the overall data density as a combination of influence functions from
individual data points.

Steps in the DENCLUE Algorithm

1. Kernel Density Estimation

o The density at a point is estimated using kernel density functions. Each data
point contributes to the overall density based on its influence function.

o Example: In a one-dimensional dataset, the density at a point is determined by

the sum of Gaussian kernels centered on each data point.

2. Density Peaks and Attractors

o The algorithm identifies local density attractors (peaks in the density

function). These peaks represent regions with the highest density in the data.

o Example: In Figure 9.13, points A, B, C, D, and E are density attractors.

3. Hill-Climbing Procedure

o Each data point is assigned to the nearest density attractor by a hill-climbing

process, where the algorithm moves iteratively toward the highest density
region.

o Example: A data point near attractor B will climb to its peak and be assigned
to the cluster around B.

4. Cluster Formation

o Data points associated with the same density attractor form a cluster.
Attractors with density below a threshold ξ are treated as noise.

o Example: Attractor C in Figure 9.13 has a density below ξ, so it is discarded as

noise.

5. Cluster Merging
 o Clusters whose density attractors are connected by a path of points with
density above ξ are merged.

o Example: Clusters D and E in Figure 9.13 are connected by a path with

density above ξ and are combined into one cluster. Clusters A and B remain
separate.

6. Cluster Shapes

o DENCLUE can detect clusters of arbitrary shapes due to its reliance on

density estimation and merging based on density paths.

Example: Clustering in One Dimension

Using a dataset of points distributed along a line:

 Peaks (Density Attractors): Points where the density is highest (e.g., A, B, D, E).

 Threshold ξ: Minimum density for a peak to form a valid cluster (e.g., discard C as
noise).

 Path-Based Merging: Combine D and E if a path connects them above ξ.

Advantages of DENCLUE

 Identifies clusters of arbitrary shapes.

 Filters out noise points using a density threshold.

 Flexible and intuitive due to the use of density functions.

Limitations

 Performance depends on the choice of kernel function and bandwidth.

 Computationally intensive for large datasets.