Introduction to Machine Learning

What is Machine Learning?

A subset of artificial intelligence known as machine learning focuses primarily on the creation of
algorithms that enable a computer to independently learn from data and previous experiences. Arthur
Samuel first used the term "machine learning" in 1959. It could be summarized as follows:

Without being explicitly programmed, machine learning enables a machine to automatically learn from
data, improve performance from experiences, and predict things.

Machine learning algorithms create a mathematical model that, without being explicitly programmed,
aids in making predictions or decisions with the assistance of sample historical data, or training data.
For the purpose of developing predictive models, machine learning brings together statistics and
computer science. Algorithms that learn from historical data are either constructed or utilized in
machine learning. The performance will rise in proportion to the quantity of information we provide.

A machine can learn if it can gain more data to improve its performance.

How does Machine Learning work

A machine learning system builds prediction models, learns from previous data, and predicts the
output of new data whenever it receives it. The amount of data helps to build a better model that
accurately predicts the output, which in turn affects the accuracy of the predicted output.

Let's say we have a complex problem in which we need to make predictions. Instead of writing code,
we just need to feed the data to generic algorithms, which build the logic based on the data and predict
the output. Our perspective on the issue has changed as a result of machine learning. The Machine
Learning algorithm's operation is depicted in the following block diagram:

Features of Machine Learning:

o Machine learning uses data to detect various patterns in a given dataset.

o It can learn from past data and improve automatically.
o It is a data-driven technology.
o Machine learning is much similar to data mining as it also deals with the huge amount of the
data.

Need for Machine Learning

The demand for machine learning is steadily rising. Because it is able to perform tasks that are too
complex for a person to directly implement, machine learning is required. Humans are constrained by
our inability to manually access vast amounts of data; as a result, we require computer systems, which
is where machine learning comes in to simplify our lives.
By providing them with a large amount of data and allowing them to automatically explore the data,
build models, and predict the required output, we can train machine learning algorithms. The cost
function can be used to determine the amount of data and the machine learning algorithm's
performance. We can save both time and money by using machine learning.

The significance of AI can be handily perceived by its utilization's cases, Presently, AI is utilized in self-
driving vehicles, digital misrepresentation identification, face acknowledgment, and companion idea
by Facebook, and so on. Different top organizations, for example, Netflix and Amazon have constructed
AI models that are utilizing an immense measure of information to examine the client interest and
suggest item likewise.

Following are some key points which show the importance of Machine Learning:

➢ Rapid increment in the production of data

➢ Solving complex problems, which are difficult for a human
➢ Decision making in various sector including finance
➢ Finding hidden patterns and extracting useful information from data.

Classification of Machine Learning

At a broad level, machine learning can be classified into three types:

➢ Supervised learning
➢ Unsupervised learning
➢ Semi-supervised learning
➢ Reinforcement learning

1) Supervised Learning

In supervised learning, sample labeled data are provided to the machine learning system for training,
and the system then predicts the output based on the training data.

The system uses labeled data to build a model that understands the datasets and learns about each
one. After the training and processing are done, we test the model with sample data to see if it can
accurately predict the output.

The mapping of the input data to the output data is the objective of supervised learning. The managed
learning depends on oversight, and it is equivalent to when an understudy learns things in the
management of the educator. Spam filtering is an example of supervised learning.
Supervised learning can be grouped further in two categories of algorithms:

➢ Classification
➢ Regression

2) Unsupervised Learning

Unsupervised learning is a learning method in which a machine learns without any supervision.

The training is provided to the machine with the set of data that has not been labeled, classified, or
categorized, and the algorithm needs to act on that data without any supervision. The goal of
unsupervised learning is to restructure the input data into new features or a group of objects with
similar patterns.

In unsupervised learning, we don't have a predetermined result. The machine tries to find useful
insights from the huge amount of data. It can be further classifieds into two categories of algorithms:

➢ Clustering
➢ Association

3) Reinforcement Learning

Reinforcement learning is a feedback-based learning method, in which a learning agent gets a reward
for each right action and gets a penalty for each wrong action. The agent learns automatically with
these feedbacks and improves its performance. In reinforcement learning, the agent interacts with the
environment and explores it. The goal of an agent is to get the most reward points, and hence, it
improves its performance.

The robotic dog, which automatically learns the movement of his arms, is an example of Reinforcement
learning.

Applications of Machine learning

Machine learning is a buzzword for today's technology, and it is growing very rapidly day by day. We
are using machine learning in our daily life even without knowing it such as Google Maps, Google
assistant, Alexa, etc. Below are some most trending real-world applications of Machine Learning:
1. Image Recognition:

Image recognition is one of the most common applications of machine learning. It is used to identify
objects, persons, places, digital images, etc. The popular use case of image recognition and face
detection is, Automatic friend tagging suggestion:

Facebook provides us a feature of auto friend tagging suggestion. Whenever we upload a photo with
our Facebook friends, then we automatically get a tagging suggestion with name, and the technology
behind this is machine learning's face detection and recognition algorithm.

It is based on the Facebook project named "Deep Face," which is responsible for face recognition and
person identification in the picture.

2. Speech Recognition

While using Google, we get an option of "Search by voice," it comes under speech recognition, and it's
a popular application of machine learning.

Speech recognition is a process of converting voice instructions into text, and it is also known as
"Speech to text", or "Computer speech recognition." At present, machine learning algorithms are
widely used by various applications of speech recognition. Google assistant, Siri, Cortana, and Alexa
are using speech recognition technology to follow the voice instructions.

3. Traffic prediction:

If we want to visit a new place, we take help of Google Maps, which shows us the correct path with
the shortest route and predicts the traffic conditions.

It predicts the traffic conditions such as whether traffic is cleared, slow-moving, or heavily congested
with the help of two ways:

Real Time location of the vehicle form Google Map app and sensors

Average time has taken on past days at the same time.

Everyone who is using Google Map is helping this app to make it better. It takes information from the
user and sends back to its database to improve the performance.

4. Product recommendations:

Machine learning is widely used by various e-commerce and entertainment companies such as
Amazon, Netflix, etc., for product recommendation to the user. Whenever we search for some product
on Amazon, then we started getting an advertisement for the same product while internet surfing on
the same browser and this is because of machine learning.

Google understands the user interest using various machine learning algorithms and suggests the
product as per customer interest.

As similar, when we use Netflix, we find some recommendations for entertainment series, movies, etc.,
and this is also done with the help of machine learning.

5. Self-driving cars:
One of the most exciting applications of machine learning is self-driving cars. Machine learning plays a
significant role in self-driving cars. Tesla, the most popular car manufacturing company is working on
self-driving car. It is using unsupervised learning method to train the car models to detect people and
objects while driving.

6. Email Spam and Malware Filtering:

Whenever we receive a new email, it is filtered automatically as important, normal, and spam. We
always receive an important mail in our inbox with the important symbol and spam emails in our spam
box, and the technology behind this is Machine learning. Below are some spam filters used by Gmail:

Content Filter

Header filter

General blacklists filter

Rules-based filters

Permission filters

Some machine learning algorithms such as Multi-Layer Perceptron, Decision tree, and Naïve Bayes
classifier are used for email spam filtering and malware detection.

7. Virtual Personal Assistant:

We have various virtual personal assistants such as Google assistant, Alexa, Cortana, Siri. As the name
suggests, they help us in finding the information using our voice instruction. These assistants can help
us in various ways just by our voice instructions such as Play music, call someone, Open an email,
Scheduling an appointment, etc.

These virtual assistants use machine learning algorithms as an important part.

These assistant record our voice instructions, send it over the server on a cloud, and decode it using
ML algorithms and act accordingly.

8. Online Fraud Detection:

Machine learning is making our online transaction safe and secure by detecting fraud transaction.
Whenever we perform some online transaction, there may be various ways that a fraudulent
transaction can take place such as fake accounts, fake ids, and steal money in the middle of a
transaction. So to detect this, Feed Forward Neural network helps us by checking whether it is a
genuine transaction or a fraud transaction.

For each genuine transaction, the output is converted into some hash values, and these values become
the input for the next round. For each genuine transaction, there is a specific pattern which gets change
for the fraud transaction hence, it detects it and makes our online transactions more secure.

9. Stock Market trading:

Machine learning is widely used in stock market trading. In the stock market, there is always a risk of
up and downs in shares, so for this machine learning's long short term memory neural network is used
for the prediction of stock market trends.

10. Medical Diagnosis:

In medical science, machine learning is used for diseases diagnoses. With this, medical technology is
growing very fast and able to build 3D models that can predict the exact position of lesions in the brain.

It helps in finding brain tumors and other brain-related diseases easily.

11. Automatic Language Translation:

Nowadays, if we visit a new place and we are not aware of the language then it is not a problem at all,
as for this also machine learning helps us by converting the text into our known languages. Google's
GNMT (Google Neural Machine Translation) provide this feature, which is a Neural Machine Learning
that translates the text into our familiar language, and it called as automatic translation.

The technology behind the automatic translation is a sequence to sequence learning algorithm, which
is used with image recognition and translates the text from one language to another language.

Machine learning Life cycle

Machine learning has given the computer systems the abilities to automatically learn without being
explicitly programmed. But how does a machine learning system work? So, it can be described using
the life cycle of machine learning. Machine learning life cycle is a cyclic process to build an efficient
machine learning project. The main purpose of the life cycle is to find a solution to the problem or
project.

Machine learning life cycle involves seven major steps, which are given below:

➢ Gathering Data
➢ Data preparation
➢ Data Wrangling
➢ Analyse Data
➢ Train the model
➢ Test the model
➢ Deployment
The most important thing in the complete process is to understand the problem and to know the
purpose of the problem. Therefore, before starting the life cycle, we need to understand the problem
because the good result depends on the better understanding of the problem.

In the complete life cycle process, to solve a problem, we create a machine learning system called
"model", and this model is created by providing "training". But to train a model, we need data, hence,
life cycle starts by collecting data.

1. Gathering Data:

Data Gathering is the first step of the machine learning life cycle. The goal of this step is to identify and
obtain all data-related problems.

In this step, we need to identify the different data sources, as data can be collected from various
sources such as files, database, internet, or mobile devices. It is one of the most important steps of the
life cycle. The quantity and quality of the collected data will determine the efficiency of the output.
The more will be the data, the more accurate will be the prediction.

This step includes the below tasks:

➢ Identify various data sources

➢ Collect data
➢ Integrate the data obtained from different sources

By performing the above task, we get a coherent set of data, also called as a dataset. It will be used in
further steps.

2. Data preparation

After collecting the data, we need to prepare it for further steps. Data preparation is a step where we
put our data into a suitable place and prepare it to use in our machine learning training.

In this step, first, we put all data together, and then randomize the ordering of data.

This step can be further divided into two processes:

Data exploration:

It is used to understand the nature of data that we have to work with. We need to understand the
characteristics, format, and quality of data.

A better understanding of data leads to an effective outcome. In this, we find Correlations, general
trends, and outliers.

Data pre-processing:

Now the next step is preprocessing of data for its analysis.

3. Data Wrangling

Data wrangling is the process of cleaning and converting raw data into a useable format. It is the
process of cleaning the data, selecting the variable to use, and transforming the data in a proper format
to make it more suitable for analysis in the next step. It is one of the most important steps of the
complete process. Cleaning of data is required to address the quality issues.
It is not necessary that data we have collected is always of our use as some of the data may not be
useful. In real-world applications, collected data may have various issues, including:

➢ Missing Values
➢ Duplicate data
➢ Invalid data
➢ Noise

So, we use various filtering techniques to clean the data.

It is mandatory to detect and remove the above issues because it can negatively affect the quality of
the outcome.

4. Data Analysis

Now the cleaned and prepared data is passed on to the analysis step. This step involves:

➢ Selection of analytical techniques

➢ Building models
➢ Review the result

The aim of this step is to build a machine learning model to analyze the data using various analytical
techniques and review the outcome. It starts with the determination of the type of the problems,
where we select the machine learning techniques such as Classification, Regression, Cluster analysis,
Association, etc. then build the model using prepared data, and evaluate the model.

Hence, in this step, we take the data and use machine learning algorithms to build the model.

5. Train Model

Now the next step is to train the model, in this step we train our model to improve its performance for
better outcome of the problem.

We use datasets to train the model using various machine learning algorithms. Training a model is
required so that it can understand the various patterns, rules, and, features.

6. Test Model

Once our machine learning model has been trained on a given dataset, then we test the model. In this
step, we check for the accuracy of our model by providing a test dataset to it.

Testing the model determines the percentage accuracy of the model as per the requirement of project
or problem.

7. Deployment

The last step of machine learning life cycle is deployment, where we deploy the model in the real-
world system.

If the above-prepared model is producing an accurate result as per our requirement with acceptable
speed, then we deploy the model in the real system. But before deploying the project, we will check
whether it is improving its performance using available data or not. The deployment phase is similar
to making the final report for a project.
In the fast-evolving era of artificial intelligence, Deep Learning stands as a cornerstone technology,
revolutionizing how machines understand, learn, and interact with complex data. At its essence, Deep
Learning AI mimics the intricate neural networks of the human brain, enabling computers to
autonomously discover patterns and make decisions from vast amounts of unstructured data. This
transformative field has propelled breakthroughs across various domains, from computer vision and
natural language processing to healthcare diagnostics and autonomous driving.

What is Deep Learning?

The definition of Deep learning is that it is the branch of machine learning that is based on artificial
neural network architecture. An artificial neural network or ANN uses layers of interconnected nodes
called neurons that work together to process and learn from the input data.

In a fully connected Deep neural network, there is an input layer and one or more hidden layers
connected one after the other. Each neuron receives input from the previous layer neurons or the
input layer. The output of one neuron becomes the input to other neurons in the next layer of the
network, and this process continues until the final layer produces the output of the network. The layers
of the neural network transform the input data through a series of nonlinear transformations, allowing
the network to learn complex representations of the input data.

Today Deep learning AI has become one of the most popular and visible areas of machine learning,
due to its success in a variety of applications, such as computer vision, natural language processing,
and Reinforcement learning.

Deep learning AI can be used for supervised, unsupervised as well as reinforcement machine learning.
it uses a variety of ways to process these.

Supervised Machine Learning: Supervised machine learning is the machine learning technique in
which the neural network learns to make predictions or classify data based on the labeled datasets.
Here we input both input features along with the target variables. the neural network learns to make
predictions based on the cost or error that comes from the difference between the predicted and the
actual target, this process is known as backpropagation. Deep learning algorithms like Convolutional
neural networks, Recurrent neural networks are used for many supervised tasks like image
classifications and recognization, sentiment analysis, language translations, etc.

Unsupervised Machine Learning: Unsupervised machine learning is the machine learning technique in
which the neural network learns to discover the patterns or to cluster the dataset based on unlabeled
datasets. Here there are no target variables. while the machine has to self-determined the hidden
patterns or relationships within the datasets. Deep learning algorithms like autoencoders and
generative models are used for unsupervised tasks like clustering, dimensionality reduction, and
anomaly detection.

Reinforcement Machine Learning: Reinforcement Machine Learning is the machine learning

technique in which an agent learns to make decisions in an environment to maximize a reward signal.
The agent interacts with the environment by taking action and observing the resulting rewards. Deep
learning can be used to learn policies, or a set of actions, that maximizes the cumulative reward over
time. Deep reinforcement learning algorithms like Deep Q networks and Deep Deterministic Policy
Gradient (DDPG) are used to reinforce tasks like robotics and game playing etc.

Artificial neural networks

Artificial neural networks are built on the principles of the structure and operation of human neurons.
It is also known as neural networks or neural nets. An artificial neural network’s input layer, which is
the first layer, receives input from external sources and passes it on to the hidden layer, which is the
second layer. Each neuron in the hidden layer gets information from the neurons in the previous layer,
computes the weighted total, and then transfers it to the neurons in the next layer. These conn ections
are weighted, which means that the impacts of the inputs from the preceding layer are more or less
optimized by giving each input a distinct weight. These weights are then adjusted during the training
process to enhance the performance of the model.

Artificial neurons, also known as units, are found in artificial neural networks. The whole Artificial
Neural Network is composed of these artificial neurons, which are arranged in a series of layers. The
complexities of neural networks will depend on the complexities of the underlying patterns in the
dataset whether a layer has a dozen units or millions of units. Commonly, Artificial Neural Network
has an input layer, an output layer as well as hidden layers. The input layer receives data from the
outside world which the neural network needs to analyze or learn about.

In a fully connected artificial neural network, there is an input layer and one or more hidden layers
connected one after the other. Each neuron receives input from the previous layer neurons or the
input layer. The output of one neuron becomes the input to other neurons in the next layer of the
network, and this process continues until the final layer produces the output of the network. Then,
after passing through one or more hidden layers, this data is transformed into valuable data for the
output layer. Finally, the output layer provides an output in the form of an artificial neural network’s
response to the data that comes in.

Units are linked to one another from one layer to another in the bulk of neural networks. Each of these
links has weights that control how much one unit influences another. The neural network learns more
and more about the data as it moves from one unit to another, ultimately producing an output from
the output layer.

Difference between Machine Learning and Deep Learning :

machine learning and deep learning AI both are subsets of artificial intelligence but there are many
similarities and differences between them.

Types of neural networks

Deep Learning models are able to automatically learn features from the data, which makes them well-
suited for tasks such as image recognition, speech recognition, and natural language processing. The
most widely used architectures in deep learning are feedforward neural networks, convolutional
neural networks (CNNs), and recurrent neural networks (RNNs).

Feedforward neural networks (FNNs) are the simplest type of ANN, with a linear flow of information
through the network. FNNs have been widely used for tasks such as image classification, speech
recognition, and natural language processing.

Convolutional Neural Networks (CNNs) are specifically for image and video recognition tasks. CNNs are
able to automatically learn features from the images, which makes them well-suited for tasks such as
image classification, object detection, and image segmentation.

Recurrent Neural Networks (RNNs) are a type of neural network that is able to process sequential data,
such as time series and natural language. RNNs are able to maintain an internal state that captures
information about the previous inputs, which makes them well-suited for tasks such as speech
recognition, natural language processing, and language translation.

Deep Learning Applications:

The main applications of deep learning AI can be divided into computer vision, natural language
processing (NLP), and reinforcement learning.

Computer vision

The first Deep Learning applications is Computer vision. In computer vision, Deep learning AI models
can enable machines to identify and understand visual data. Some of the main applications of deep
learning in computer vision include:

Object detection and recognition: Deep learning model can be used to identify and locate objects
within images and videos, making it possible for machines to perform tasks such as self-driving cars,
surveillance, and robotics.

Image classification: Deep learning models can be used to classify images into categories such as
animals, plants, and buildings. This is used in applications such as medical imaging, quality control, and
image retrieval.

Image segmentation: Deep learning models can be used for image segmentation into different regions,
making it possible to identify specific features within images.

2. Natural language processing (NLP):

In Deep learning applications, second application is NLP. NLP, the Deep learning model can enable
machines to understand and generate human language. Some of the main applications of deep
learning in NLP include:

Automatic Text Generation – Deep learning model can learn the corpus of text and new text like
summaries, essays can be automatically generated using these trained models.

Language translation: Deep learning models can translate text from one language to another, making
it possible to communicate with people from different linguistic backgrounds.
Sentiment analysis: Deep learning models can analyze the sentiment of a piece of text, making it
possible to determine whether the text is positive, negative, or neutral. This is used in applications
such as customer service, social media monitoring, and political analysis.

Speech recognition: Deep learning models can recognize and transcribe spoken words, making it
possible to perform tasks such as speech-to-text conversion, voice search, and voice-controlled
devices.

3. Reinforcement learning:

In reinforcement learning, deep learning works as training agents to take action in an environment to
maximize a reward. Some of the main applications of deep learning in reinforcement learning include:

Game playing: Deep reinforcement learning models have been able to beat human experts at games
such as Go, Chess, and Atari.

Robotics: Deep reinforcement learning models can be used to train robots to perform complex tasks
such as grasping objects, navigation, and manipulation.

Control systems: Deep reinforcement learning models can be used to control complex systems such as
power grids, traffic management, and supply chain optimization.

Challenges in Deep Learning

Deep learning has made significant advancements in various fields, but there are still some challenges
that need to be addressed. Here are some of the main challenges in deep learning:

Data availability: It requires large amounts of data to learn from. For using deep learning it’s a big
concern to gather as much data for training.

Computational Resources: For training the deep learning model, it is computationally expensive
because it requires specialized hardware like GPUs and TPUs.

Time-consuming: While working on sequential data depending on the computational resource it can
take very large even in days or months.

Interpretability: Deep learning models are complex, it works like a black box. it is very difficult to
interpret the result.

Overfitting: when the model is trained again and again, it becomes too specialized for the training data,
leading to overfitting and poor performance on new data.

Advantages of Deep Learning:

High accuracy: Deep Learning algorithms can achieve state-of-the-art performance in various tasks,
such as image recognition and natural language processing.

Automated feature engineering: Deep Learning algorithms can automatically discover and learn
relevant features from data without the need for manual feature engineering.

Scalability: Deep Learning models can scale to handle large and complex datasets, and can learn from
massive amounts of data.
Flexibility: Deep Learning models can be applied to a wide range of tasks and can handle various types
of data, such as images, text, and speech.

Continual improvement: Deep Learning models can continually improve their performance as more
data becomes available.

Disadvantages of Deep Learning:

High computational requirements: Deep Learning AI models require large amounts of data and
computational resources to train and optimize.

Requires large amounts of labeled data: Deep Learning models often require a large amount of labeled
data for training, which can be expensive and time- consuming to acquire.

Interpretability: Deep Learning models can be challenging to interpret, making it difficult to understand
how they make decisions.

Overfitting: Deep Learning models can sometimes overfit to the training data, resulting in poor
performance on new and unseen data.

Black-box nature: Deep Learning models are often treated as black boxes, making it difficult to
understand how they work and how they arrived at their predictions.

Conclusion
In conclusion, the field of Deep Learning represents a transformative leap in artificial intelligence. By
mimicking the human brain’s neural networks, Deep Learning AI algorithms have revolutionized
industries ranging from healthcare to finance, from autonomous vehicles to natural language
processing. As we continue to push the boundaries of computational power and dataset sizes, the
potential applications of Deep Learning are limitless. However, challenges such as interpretability and
ethical considerations remain significant. Yet, with ongoing research and innovation, Deep Learning
promises to reshape our future, ushering in a new era where machines can learn, adapt, and solve
complex problems at a scale and speed previously unimaginable.
Linear Regression

What is Linear Regression?

Linear regression is a type of supervised machine learning algorithm that computes the linear
relationship between the dependent variable and one or more independent features by fitting a
linear equation to observed data.

When there is only one independent feature, it is known as Simple Linear Regression, and when
there are more than one feature, it is known as Multiple Linear Regression.

Similarly, when there is only one dependent variable, it is considered Univariate Linear Regression,
while when there are more than one dependent variables, it is known as Multivariate Regression.

Why Linear Regression is Important?

The interpretability of linear regression is a notable strength. The model’s equation provides clear
coefficients that elucidate the impact of each independent variable on the dependent variable,
facilitating a deeper understanding of the underlying dynamics. Its simplicity is a virtue, as linear
regression is transparent, easy to implement, and serves as a foundational concept for more
complex algorithms.

Linear regression is not merely a predictive tool; it forms the basis for various advanced models.
Techniques like regularization and support vector machines draw inspiration from linear
regression, expanding its utility. Additionally, linear regression is a cornerstone in assumption
testing, enabling researchers to validate key assumptions about the data.

Types of Linear Regression

There are two main types of linear regression:

Simple Linear Regression

This is the simplest form of linear regression, and it involves only one independent variable and
one dependent variable. The equation for simple linear regression is:

+
β

y=β

+β

where:

Y is the dependent variable

X is the independent variable

β0 is the intercept

β1 is the slope

Multiple Linear Regression

This involves more than one independent variable and one dependent variable. The equation for
multiple linear regression is:
where:

Y is the dependent variable

X1, X2, …, Xn are the independent variables

β0 is the intercept

β1, β2, …, βn are the slopes

The goal of the algorithm is to find the best Fit Line equation that can predict the values based on
the independent variables.

In regression set of records are present with X and Y values and these values are used to learn a
function so if you want to predict Y from an unknown X this learned function can be used. In
regression we have to find the value of Y, So, a function is required that predicts continuous Y in the
case of regression given X as independent features.

What is the best Fit Line?

Our primary objective while using linear regression is to locate the best-fit line, which implies that
the error between the predicted and actual values should be kept to a minimum. There will be the
least error in the best-fit line.

The best Fit Line equation provides a straight line that represents the relationship between the
dependent and independent variables. The slope of the line indicates how much the dependent
variable changes for a unit change in the independent variable(s).
Logistic regression

Logistic regression is a supervised machine learning algorithm used for classification tasks where the
goal is to predict the probability that an instance belongs to a given class or not. Logistic regression is
a statistical algorithm which analyze the relationship between two data factors. The article explores
the fundamentals of logistic regression, it’s types and implementations.

What is Logistic Regression?

Logistic regression is used for binary classification where we use sigmoid function, that takes input as
independent variables and produces a probability value between 0 and 1.

For example, we have two classes Class 0 and Class 1 if the value of the logistic function for an input is
greater than 0.5 (threshold value) then it belongs to Class 1 otherwise it belongs to Class 0. It’s referred
to as regression because it is the extension of linear regression but is mainly used for classification
problems.

Key Points:

Logistic regression predicts the output of a categorical dependent variable. Therefore, the outcome
must be a categorical or discrete value.

It can be either Yes or No, 0 or 1, true or False, etc. but instead of giving the exact value as 0 and 1, it
gives the probabilistic values which lie between 0 and 1.

In Logistic regression, instead of fitting a regression line, we fit an “S” shaped logistic function, which
predicts two maximum values (0 or 1).

Logistic Function – Sigmoid Function

The sigmoid function is a mathematical function used to map the predicted values to probabilities.

It maps any real value into another value within a range of 0 and 1. The value of the logistic regression
must be between 0 and 1, which cannot go beyond this limit, so it forms a curve like the “S” form.

The S-form curve is called the Sigmoid function or the logistic function.

In logistic regression, we use the concept of the threshold value, which defines the probability of either
0 or 1. Such as values above the threshold value tends to 1, and a value below the threshold values
tends to 0.

Types of Logistic Regression

On the basis of the categories, Logistic Regression can be classified into three types:

Binomial: In binomial Logistic regression, there can be only two possible types of the dependent
variables, such as 0 or 1, Pass or Fail, etc.

Multinomial: In multinomial Logistic regression, there can be 3 or more possible unordered types of
the dependent variable, such as “cat”, “dogs”, or “sheep”
Ordinal: In ordinal Logistic regression, there can be 3 or more possible ordered types of dependent
variables, such as “low”, “Medium”, or “High”.

Assumptions of Logistic Regression

We will explore the assumptions of logistic regression as understanding these assumptions is

important to ensure that we are using appropriate application of the model. The assumption include:

Independent observations: Each observation is independent of the other. meaning there is no

correlation between any input variables.

Binary dependent variables: It takes the assumption that the dependent variable must be binary or
dichotomous, meaning it can take only two values. For more than two categories SoftMax functions
are used.

Linearity relationship between independent variables and log odds: The relationship between the
independent variables and the log odds of the dependent variable should be linear.

No outliers: There should be no outliers in the dataset.

Large sample size: The sample size is sufficiently large

Terminologies involved in Logistic Regression

Here are some common terms involved in logistic regression:

Independent variables: The input characteristics or predictor factors applied to the dependent
variable’s predictions.

Dependent variable: The target variable in a logistic regression model, which we are trying to predict.

Logistic function: The formula used to represent how the independent and dependent variables relate
to one another. The logistic function transforms the input variables into a probability value between 0
and 1, which represents the likelihood of the dependent variable being 1 or 0.

Odds: It is the ratio of something occurring to something not occurring. it is different from probability
as the probability is the ratio of something occurring to everything that could possibly occur.

Log-odds: The log-odds, also known as the logit function, is the natural logarithm of the odds. In logistic
regression, the log odds of the dependent variable are modeled as a linear combination of the
independent variables and the intercept.

Coefficient: The logistic regression model’s estimated parameters, show how the independent and
dependent variables relate to one another.

Intercept: A constant term in the logistic regression model, which represents the log odds when all
independent variables are equal to zero.

Maximum likelihood estimation: The method used to estimate the coefficients of the logistic
regression model, which maximizes the likelihood of observing the data given the model.

How does Logistic Regression work?

The logistic regression model transforms the linear regression function continuous value output into
categorical value output using a sigmoid function, which maps any real-valued set of independent
variables input into a value between 0 and 1. This function is known as the logistic function.

Let the independent input features be:

Association Rule Learning
Association rule learning is a type of unsupervised learning technique that checks for the dependency
of one data item on another data item and maps accordingly so that it can be more profitable. It tries
to find some interesting relations or associations among the variables of dataset. It is based on
different rules to discover the interesting relations between variables in the database.

The association rule learning is one of the very important concepts of machine learning, and it is
employed in Market Basket analysis, Web usage mining, continuous production, etc. Here market
basket analysis is a technique used by the various big retailer to discover the associations between
items. We can understand it by taking an example of a supermarket, as in a supermarket, all products
that are purchased together are put together.

For example, if a customer buys bread, he most likely can also buy butter, eggs, or milk, so these
products are stored within a shelf or mostly nearby. Consider the below diagram:

Association rule learning can be divided into three types of algorithms:

Apriori
Eclat
F-P Growth Algorithm
How does Association Rule Learning work?

Association rule learning works on the concept of If and Else Statement, such as if A then B.
Here the If element is called antecedent, and then statement is called as Consequent. These types of
relationships where we can find out some association or relation between two items is known as single
cardinality. It is all about creating rules, and if the number of items increases, then cardinality also
increases accordingly. So, to measure the associations between thousands of data items, there are
several metrics. These metrics are given below:

o Support

o Confidence

o Lift

Let's understand each of them:

Support

Support is the frequency of A or how frequently an item appears in the dataset. It is defined as the
fraction of the transaction T that contains the itemset X. If there are X datasets, then for transactions
T, it can be written as:

Confidence

Confidence indicates how often the rule has been found to be true. Or how often the items X and Y
occur together in the dataset when the occurrence of X is already given. It is the ratio of the transaction
that contains X and Y to the number of records that contain X.

Lift

It is the strength of any rule, which can be defined as below formula:

It is the ratio of the observed support measure and expected support if X and Y are independent of
each other. It has three possible values:

o If Lift= 1: The probability of occurrence of antecedent and consequent is independent of each

other.

o Lift>1: It determines the degree to which the two itemsets are dependent to each other.
 o Lift<1: It tells us that one item is a substitute for other items, which means one item has a
negative effect on another.

Association rule learning can be divided into three algorithms:

Apriori Algorithm

This algorithm uses frequent datasets to generate association rules. It is designed to work on the
databases that contain transactions. This algorithm uses a breadth-first search and Hash Tree to
calculate the itemset efficiently.

It is mainly used for market basket analysis and helps to understand the products that can be bought
together. It can also be used in the healthcare field to find drug reactions for patients.

Eclat Algorithm

Eclat algorithm stands for Equivalence Class Transformation. This algorithm uses a depth-first search
technique to find frequent itemsets in a transaction database. It performs faster execution than Apriori
Algorithm.

F-P Growth Algorithm

The F-P growth algorithm stands for Frequent Pattern, and it is the improved version of the Apriori
Algorithm. It represents the database in the form of a tree structure that is known as a frequent pattern
or tree. The purpose of this frequent tree is to extract the most frequent patterns.

Applications of Association Rule Learning

It has various applications in machine learning and data mining. Below are some popular applications
of association rule learning:

o Market Basket Analysis: It is one of the popular examples and applications of association rule
mining. This technique is commonly used by big retailers to determine the association
between items.

o Medical Diagnosis: With the help of association rules, patients can be cured easily, as it helps
in identifying the probability of illness for a particular disease.

o Protein Sequence: The association rules help in determining the synthesis of artificial Proteins.

o It is also used for the Catalog Design and Loss-leader Analysis and many more other
applications.
Association Rule

Association rule mining finds interesting associations and relationships among large sets of data items.
This rule shows how frequently a itemset occurs in a transaction. A typical example is a Market Based
Analysis. Market Based Analysis is one of the key techniques used by large relations to show
associations between items.It allows retailers to identify relationships between the items that people
buy together frequently. Given a set of transactions, we can find rules that will predict the occurrence
of an item based on the occurrences of other items in the transaction.

TID Items

1 Bread, Milk

2 Bread, Diaper, Beer, Eggs

3 Milk, Diaper, Beer, Coke

4 Bread, Milk, Diaper, Beer

5 Bread, Milk, Diaper, Coke

Before we start defining the rule, let us first see the basic definitions. Support Count( ) – Frequency
of occurrence of a itemset.