Scribd
Upload
4 views

naive_bayes

Naïve Bayes is a probabilistic classification model that uses Bayes' Theorem to predict class labels based on the highest probability given input features, assuming feature independence. The model simplifies the estimation of probabilities by leveraging the independence assumption, allowing for efficient classification. Despite its assumptions, Naïve Bayes often performs well in practice, even when the independence assumption does not hold true.

Uploaded by

dothtrung4897
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
4 views

naive_bayes

Naïve Bayes is a probabilistic classification model that uses Bayes' Theorem to predict class labels based on the highest probability given input features, assuming feature independence. The model simplifies the estimation of probabilities by leveraging the independence assumption, allowing for efficient classification. Despite its assumptions, Naïve Bayes often performs well in practice, even when the independence assumption does not hold true.

Uploaded by

dothtrung4897
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 19

Naïve Bayes

• Discuss how a Naïve Bayes model works


for classification
• Define the components of Bayes’ Rule
• Explain what the ‘naïve’ means in Naïve
Bayes
Naïve Bayes Overview
• Probabilistic approach to
classification
• Relationships between input
features and class expressed as
probabilities
• Label for sample is class with
highest probability given input
Naïve Bayes Classifier
Classification
Using
Probability

Bayes
Theorem

Feature
Independence
Assumption
Probability of Event
Probability is measure of how
likely an event is

Probability of Event ‘A’ Occurring

# ways for A
P(A) =
# possible outcomes
Probability of Event
What is the probability of rolling a
die and getting 6?

Probability of Rolling 6 on a Die

# ways for getting 6 1


P(6) = =
# possible outcomes 6
Joint Probability
Probability of events A and B occurring together

P(A) P(B)

P(A,B)

Joint Probability of A and B


Joint Probability Example
What is the probability of two 6’s
when rolling two dice?

Probability of Rolling Two 6’s

1 1 1
P(A,B) = P(A) * P(B) = * =
6 6 36
Conditional Probability
Probability of event A
P(A) P(B) occurring, given that
event B occurred
P(A,B)

P(A,B) Conditional
P(A | B) =
P(B) Probability
Bayes’ Theorem
• Relationship between P(B | A) and P(A | B)
can be expressed through Bayes’ Theorem

P(A | B) * P(B)
P(B | A) =
P(A)
Bayes’ Theorem
Classification with Probabilities
Given features X={X1,X2,…,Xn}, predict class C
Do this by finding value of C that maximizes P(C | X)

P(C1|X)
P(C2|X)
. max
Class
. Label
.
P(Ck|X)
Bayes’ Theorem for Classification
• But estimating P(C|X) is difficult
• Bayes’ Theorem to the rescue!
• Simplifies problem
Bayes’ Theorem for Classification
Class-
Posterior Conditional Prior
Probability Probability Probability

P(X | C) * P(C)
P(C | X) =
P(X)

Probability of observing
values for input features
Bayes’ Theorem for Classification
Need to
Can be estimated from data!
calculate this

P(X | C) * P(C)
P(C | X) =
P(X)

Constant (can be ignored)


To get P(C | X), only need to find P(X | C) and
P(C), which can be estimated from the data!
Estimating P(C)

P( ) = 4/10 = 0.4

P( ) = 6/10 = 0.6

To estimate P(C), calculate fraction of


samples for class C in training data.
Estimating P(X | C)
Independence Assumption
• Features are independent of one another:
P(X1,X2,…,Xn | C) = P(X1|C) * P(X2 | C) * … * P(Xn | C)

To estimate P(X | C), only need to estimate


P(Xi | C) individually  Much simpler!
Home Marital Loan Estimating P(Xi | C)
Owner Status Default

Yes Single No P(Home Owner = Yes | No) = 3/7 = 0.43


No Married No
No Single No P(Marital Status = Single| Yes) = 2/3 = 0.67
Yes Married No
No Divorced Yes
No Married No
Yes Divorced No
No Single Yes
No Married No
No Single Yes
Source: http://www-users.cs.umn.edu/~kumar/dmbook/index.php
Naïve Bayes Classification
• Fast and simple
• Scales well
• Independence assumption
may not hold true
• In practice, still works quite well
• Does not model interactions
between features
Naïve Bayes Classifier
Classification
Using
Probability

Bayes
Theorem

Feature
Independence
Assumption

You might also like