See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/369067312

Naïve Bayes Algorithms

Research Proposal · February 2023

DOI: 10.13140/RG.2.2.15378.73921

CITATION READS
1 1,134

2 authors, including:

Mariam Alahmar
Prince Sattam Bin Abdulaziz University
4 PUBLICATIONS 3 CITATIONS

SEE PROFILE

All content following this page was uploaded by Mariam Alahmar on 08 March 2023.

The user has requested enhancement of the downloaded file.

 Antifiction Intelligent project
Class: 1098

Naïve Bayes
Algorithms
Researcher:
1.Mariam Isam Alahmar 441052039
2.Leena Abdullah 441050282
3.Hadeel Abdullah 441051286
4.Raghad Fahad 442050062
5.Lama Abdullah 442051227

Supervisor:
Dr. Thavavel Vaiyapuri Thangam
Abstract: As one of the most often used occurrence influences how likely it is that
machine learning techniques, the Naive the event will occur. With Nave Bayes, the
Bayes classifier simplifies the learning method operates under the "nave"
process for us by assuming that the features presumption that every feature in a dataset is
are independent of a certain class. This unrelated to every other feature. As an
algorithm is distinguished by its speed in illustration, suppose you wish to create a
processing and efficiency in prediction classifier that can decide whether an email is
operations. In this post, we will discover the spam or not. You may extract attributes like
definition of the Naive Bayes algorithm, its the presence of specific terms or phrases, the
mathematical formula, its benefits and sender's email address, and the email subject
drawbacks, and the rationale for its line using a dataset of emails that have been
application. Finally, we shall learn about classified as spam or not. These
examples from our everyday experience. characteristics would then be used by the
Naive Bayes algorithm to determine whether
A. Introduction a new email is likely to be spam or not.
In a nutshell, it is the Nave Bayes algorithm. shown in Figure.1.
It is a probabilistic machine learning A probabilistic classification algorithm
technique that is applied to classification called Naive Bayes determines predictions
jobs and is based on Bayes' theory. The by estimating the likelihood of each class
Bayes model offers several benefits, based on the data provided as input. It
including the capacity to handle massive makes the sometimes-incorrect assumption
amounts of data and the simplicity of design that the features are independent of one
and construction. another, although this simplification enables
With these properties, it surpasses a number accurate calculations. The probabilities of
of more sophisticated and complex each feature given each class are taught to
algorithms, but why is it called Is that naive? the algorithm by training on labeled data.
What is the cause of its increased Afterward, it determines the probability of
popularity? Because it presumes that the each class given the observed features and
features that make up the model are selects the class with the highest probability
independent of one another, it has been as the prediction when given fresh,
dubbed "naive." It is particularly well-liked unlabeled data.
because, while being straightforward and
quick to construct, the Nave Bayes model
performs remarkably well and is frequently
employed, largely because of its capacity to
manage data. Due to its size, it is regarded
as the preferable method for real-world
applications where prompt user responses
are essential.

B. Definition
A machine-learning algorithm called Naive
Bayes is employed for categorization jobs. It
is founded on the Bayes theorem, which
claims that prior knowledge of
circumstances that might be connected to an
Figure.1
Another illustration: If a shape has a diameter of D. Formulas
about 4 cm, is red, and is a circle, it will
integrate these three distinct properties to Although it would be difficult, the
identify the shape as an apple! But in reality, it is conditional probability may be determined
a tomato and not at all like that. The most using the joint probability. From P(c), P(x),
popular probability distributions that may be
utilized with the approach are the GaussianNB
and P(x|c), the posterior probability, P(c|x),
and MultinomialNB ones. The GaussianNB can be calculated using the Bayes theorem.
works well with continuous data, while the The nave Bayes classifier makes the
MultinomialNB works well with discrete data. assumption that the impact of a predictor's
Overall, Nave Bayes is a straightforward but value (x) on a certain class (c) is unrelated to
effective algorithm that is frequently used in the values of other predictors. This premise
spam filtering and text classification. is known as "class conditional
independence."
C. Diagram The Bayes Theorem calculation can be
expressed in the following manner:
(Observe Figure.3)
Flowchart diagram: P (c | x) is equal to P (c | x)*P (c)/P. (x)

Figure.3

• The posterior probability of a class

(target) given a predictor is denoted
by P(c|x) (attribute).
• The prior probability of the class is
P(c).
• The likelihood, or P(x|c), measures
the likelihood of a predictor for a
given class.
• P(x) stands for the predictor's prior
probability.

Figure.2
Simple illustration of an earlier email is Step 2: Create a likelihood table by
shown in Figure 4. calculating probabilities such as the
possibility of an overcast day being 0.29 and
the probability of playing being 0.64.
display in Table.3
Likelihood Table
Weather No Yes
Sunny 2 3 =5/14 0.36
Figure.4
Rainy 3 2 =5/14 0.36
Overcast - 4 =4/14 0.29
E. Algorithms Work
All 5 9
utilizing an illustration, comprehend it. I've
=5/14 =9/14
included a training data set for the weather
0.36 0.64
and its associated objective variable,
Table.3
"Study," below (suggesting possibilities of
Studying). We must now categorize whether Step 3: Next, compute the posterior
students will study or not dependent on the probability for each class using the Naive
weather. Let's carry it out by following the Bayesian equation. The result of the
steps below. prediction is the class with the highest
Step1: Create a frequency table from the posterior probability.
data. (Displayed in Tables1, 2)
Info Table Problem: Problem: If it's sunny outside,
weather study students won't study. Is this a true
Sunny No statement?
Overcast Yes
Rainy Yes P(Yes | Sunny) is equal to P(Yes | Sunny) *
Sunny Yes P(Yes) / P (Sunny)
Sunny Yes Here, the numerator is P(Sunny | Yes)*
Overcast Yes P(Yes), and the denominator is P(Sunny).
Rainy No P (Sunny |Yes) = 3/9 = 0.33, P (Sunny) =
Rainy No 5/14 = 0.36, and P (Yes) = 9/14 = 0.64 are
Sunny Yes the values in this case.
Rainy Yes The probability is now larger at P (Yes |
Sunny No Sunny) = 0.33 * 0.64 / 0.36 = 0.60.
Overcast Yes
Overcast Yes A similar approach is used by the Naive
Rainy No Bayes to forecast the probability of various
Table.1 classes based on various attributes. When
Frequency Table there are issues with several classes, this
Weather No Yes approach is primarily employed in text
Sunny 2 3 classification.
Rainy 3 2
Overcast - 4
TOTAL 5 9
Table.2
 F. The Pros and Cons Disadvantages:

Advantages: • Naive Bayes makes the uncommon

but unfounded assumption that all
• It is straightforward and simple to predictors (or features) are
use. independent. This restricts the
algorithm's usability in practical
• Less training data are necessary. usage cases.

• It handles continuous and discrete • Because some of its estimates may

data management. be off, you shouldn't take its
probability outputs too seriously.
• Given the quantity of predictors and
data points, it is quite callable. • If a categorical variable (in the test
data set) has a category that wasn't
• It may be used to create predictions included in the training data set, the
in real time and is quick. model will assign a 0 (zero)
probability and be unable to predict
• It doesn't respond to unimportant anything. This is commonly referred
aspects. to as Zero Frequency. We can
utilize the smoothing method to
• Class of test data set prediction is resolve this. Laplace estimation is
quick and simple. among the simplest smoothing
techniques.
• The classifier performs better than
other machine learning models like Naive Bayes, then, is a probabilistic
logistic regression or decision trees classification algorithm that generates
and requires less training data when predictions by estimating the likelihood of
the assumption of independence is each class based on the input data. It makes
true. the sometimes-incorrect assumption that the
features are independent of one another,
• Compared to a numerical variable, it although this simplification enables
performs well with categorical input accurate calculations. The system learned
variables (s). It is assumed that the probabilities of each feature given each
numerical variables have a normal class by training on labeled data. Afterward,
distribution (bell curve, which is a it determines the probability of each class
strong assumption). given the observed features and selects the
class with the highest probability as the
prediction when given fresh, unlabeled
data.
 G. Difference Between Naive Bayes VS logistic Regression

Compare Table

Naive Bayes Classification Logistic Regression

Based on the Bayes theorem, this classification A sample's likelihood inside a class is taken into
approach determines the feature that is most consideration, and the sample is linearly classified
important within the classification by calculating the depending on the probability. Finding the boundary
probability of each feature using the attributes that are between two or more classes and their samples so that
provided. The Bayes classification is thus given the the classes can be divided based on behavior is by far the
term Naive. most important step in this process.

This generative model targets goal B with feature A This discriminative model calculates probability directly
so that the theorem P(b|a) can be used to compute the by mapping A to B or B to A so that we may determine
probability between the two. This makes it clear if B occurred at a specific time interval as a result of A.
whether A has occurred whenever B has occurred,
making classification simple.

Since each characteristic is thought to be independent, The features are split in a linear fashion so that even if
categorization is done in a generated way. The the features are correlated, due to linear classification,
classification will not take place in the way that is logistic regression works in favor of data analysis and
anticipated if any of the traits are connected. gives better results than Naive Bayes.

We can perform better classification based on the Less data is not in favor of logistic regression because
number of features contributing to good probability of the resulting model with the given features will be more
the data even before the data analysis when the total broad. Regression techniques will help to reduce
amount of data taken into account is small or the overfitting, but the outcome will not be what was
sample size is less. This supports market forecasting anticipated, and analysis will not aid in understanding the
indirectly, assisting analysts in identifying the data.
standout feature.

Naive Bayes has a lower variance and a bigger bias. Low bias and increased variance characterize logistic
Results are studied to understand the data production, regression. With categorical and continuous variables,
making predictions with fewer variables and less the probability is predicted in an indirect functional
information easier. When only a small number of manner, resulting in a collection of categorical results.
training sets are taken into account, naive bayes Multi-class logistic regression is used for data analysis
provides a speedier solution. when there are multiple classes.
Table.4
We conclude from the previous table
(Table.4) these points: • If the classification is performed on a
little amount of data and there are
dependent features that were
• When we have correlated features disregarded during the algorithmic
for both Naive Bayes and logistic computation, the error is larger in
regression, correlation happens with Naive Bayes, making it a serious
labels by creating predictions; as a error. As a result, Nave Bayes is not
result, when the labels repeat, there always the best option for
are greater chances for the repetitive classification issues. Logistic
features to be highlighted in the regression has a lower error rate and
Naive Bayes method. This won't makes it simple to identify
occur with a logistic regression dependent or independent features
model since the recurring traits are from huge data sets.
counted less frequently,
compensating for the repetition. • When constructing assumptions in
logistic regression, the training data
• Naive Bayes cannot directly is taken into immediate
calculate the entries of features consideration. Naive Bayes
because there is no optimization. classification uses a tiny sample
However, we are unable to add rather than directly using training
different features to this issue. data. A discriminative classifier,
Aiming for greater perfection, naive logistic regression discriminates the
bayes calculates exclusively from the target value for any given input
features but produces subpar results values. The Naive Bayes method
when the number of features takes into account every attribute.
increases. It does not take
calibrations into account, and if there
is a dependency between features, it H. Benefits
does take it into account and
increases the feature's prominence. If An underlying presumption of a Nave Bayes
a feature has a negative effect, the classifier is that the presence of one feature
results will be subpar. This is not a in a class has no influence on the presence
problem in logistic regression since of any other characteristics. When you
the characteristics are calibrated consider that this property's existence is
precisely when they are introduced unrelated to the existence of any other
more frequently, yielding accurate property, I mean that you believe the
results. properties between them to be independent
of one another. For instance, if you see a
• Naive Bayes counts each class circle that is red, 3 inches in diameter, and
separately and bases its results on the has those three characteristics combined,
class with the highest feature count. there's a chance that it's an apple. You see, if
In logistic regression, the classes are this algorithm identifies the properties, it
divided, making it possible to find means that it is more likely to be what I
the dominant feature based on desire, which is why it was given the name
calibration. "Naive."
Why do we use it? I. Applications
Because it provides a quick and highly
scalable model for building and scoring, the In reality, we employ this technique for a
Naive Bayes method is what we utilize. It variety of tasks, including real-world
has a well-known linear scaling relationship problem-solving.
with the number of predictors and rows. So in addition to the instances provided
Additionally, it is addressing classification previously, we will present some further
issues. A Naive Bayes classifier outperforms Among these usage are the following:
other models when the assumption of
independent predictors is valid. • Shopping Example
Additionally, we can use Naive Bayes to the To use a Naive Bayes classifier to forecast
following situations: my likelihood of making a purchase based
on a given day, discount, and free delivery.
▪ Face Recognition seen in Figure 5.
It serves as a classifier by identifying faces
or other facial features like the nose, lips,
and eyes.

▪ Weather Prediction
It can be used to foretell whether the
weather will be favorable or unfavorable.
If the weather looks promising up front, you Figure.5
can use the Naive Bayes algorithm to predict
the weather.
• Trip on a train
▪ Medical Diagnosis If there is a delay and I need the outcome
With the help of the classifier's knowledge, right away, I need to know if you should
doctors may diagnose patients. Naive Bayes board the train right away or wait. This
can be used by medical experts to determine algorithm is being followed because it
whether a patient has a high risk of enables me to extract findings more quickly.
contracting a certain illness, such as cancer
or heart disease. • Group of articles
When I have a collection of articles, I like to
▪ News Classification sort them into categories such as those that
Google News determines whether the news are specifically about sports or those that are
is political, foreign news, etc. with the aid of about cuisine so that I can quickly
a Naive Bayes classifier. categorize each one.
Since the Naive Bayes Classifier has so
many uses, understanding how it functions
• Spam filtering: (Email)
is important.
Now that we have this algorithm, we can
directly see whether an email is spam or not.
 J. Application of Naïve Bayes in
python using
• Text classification:
One of the most well-known applications of First app:
this algorithm is text classification, which Python Naive Bayes Classifier from Scratch
makes use of it as a probabilistic learning (take data from Python library).
technique and applies it to documents
belonging to one or more classes. Step 1: Make a dataset.
• Sentiment Analysis:
To decide if the feelings are positive or
negative, the program will examine them.

• Recommendation System:
The naive Bayesian algorithm is a collection
of collaborative outgoing filtering used to
create mixed recommendation systems that
assist the user in determining whether or not
to accept any given resource.

Summering it will show in Figure.6

Face
Recognitio
n Step 2: Calculate the prior and conditional
Spam Weather probabilities before training the Naive Bayes
filtering Prediction model.

Applicatio
n
sentiment Medical
analysis Diagnosis

Text
classificatio Step 3: Make a prediction.
n

Figure.6
Second app:
Step 1: Separate by class:
Python Naive Bayes Classifier from Scratch
(Iris Flower Specie).
We must determine the probability of data
according to the class to which they belong,
With measurements of iris flowers as a or the so-called base rate.
starting point, the Iris Flower Dataset
predicts the species of flower. This implies that we must first divide our
training data into classes. simple in its
operation.
It is an issue in multiclass classification.
There are an equal number of observations A list of all the records can be added as the
for each class. With four input variables and dictionary's value once we establish a
dictionary object in which each key is the
one outcome variable, there are 150
class value.
observations. The names of the variables are
as follows:

• Petal length in centimeters

• Sepal breadth in centimeters;
• Petal width in centimeters
Putting data together:
• Class

About 33% of the problem is performed at

baseline.

Figure.7

First visit:
https://www.onlinegdb.com/online_python_
compiler, then follow the instructions below.
Step 2: Summarize dataset:

From a given set of data, we require two

statistics. In the next phases, we'll see how
these statistics are used to calculate
probabilities. We need the mean and
standard deviation from a given dataset as
the two statistics (average deviation from the
mean).

The formula for calculating the mean, which

Step 3: Summarize data by class:
is the average value, is:

The sample standard deviation is calculated

using the mean departure from the mean
value. This formula can be used to compute
standard deviation:

• sqrt((sum I to N (x i - mean(x))2) /
N-1)

You can see that we calculate the average

squared difference from the mean, square
the difference between the mean and the
given value, and then take the square root to
restore the units to their initial value.
Step 4: Gaussian probability density K. Summary:
function:
In this article, the algorithm was defined in
detail, explained, and supported by both
technical and real-world examples. The
algorithm was then compared to other
algorithms, and as researchers, we were able
to draw out its advantages and
disadvantages. We then applied the
algorithm in the Python programming
language to achieve the desired results.

L. conclusion:
Step 5: Class probabilities:
We have arrived at a powerful yet
straightforward method that can be used to
many different situations. When utilized
properly, the algorithm is inexpensive to
compute, quick, and extremely accurate.
Additionally, it may be trained on tiny
datasets and does not require a huge one.
Naive Bayes has been used for a variety of
applications, including sentiment analysis,
spam filtering, and text classification. It is
based on the Bayes theorem and assumes
naively that features are independent.
Despite this presumption, it has been
demonstrated to perform well on a range of
tasks and is a preferred option in machine
learning and natural language processing
because of its ease of use, scalability, and
effective performance. It is highly helpful
and widely utilized.
 M. Reference:
➢ Articles ➢ Website
• (accessed Jan. 10, 2023). • http://www.saedsayad.com/naive
• “(PDF) An Empirical Study of _bayesian.htm
the Naïve Bayes Classifier.” • https://towardsdatascience.com/i
• (accessed Jan. 10, 2023). ntroduction-to-na%C3%AFve-
• “Naïve Bayes Algorithm in ML: bayes-classifier-fa59e3e24aaf
Simplifying Classification • https://scikit-
Problems.” learn.org/stable/modules/generat
https://www.turing.com/kb/an- ed/sklearn.naive_bayes.Gaussian
introduction-to-naive-bayes-
NB.html
algorithm-for-beginners
(accessed Jan. 11, 2023). • https://www.semanticscholar.org/
• L. Setruk, “Naïve Bayes paper/Design-and-Development-
Classifier (with of-Naive-Bayes-Classifier-
examples),” Medium, Mar. 24, Garg/ebc80ed6d2d0a96ae46c995
2021. c408dd8895c4c6a19
https://leasetruk.medium.com/nai
ve-bayes-classifier-with- • https://www.researchgate.net/pub
examples-7b541f9ffedf (accessed lication/228845263_An_Empiric
Jan. 11, 2023). al_Study_of_the_Naive_Bayes_
Classifier
• https://saudidatasci.wordpress.co
m/2016/09/03/bayes-theorem/
• https://www.youtube.com/watch?
v=LGigBMtwygg

➢ other:
• Book: Machine Learning, 1997.
• Graduation CS friends from 2019

On demo day, we'll see what application has produced! Wait for us soon

View publication stats