Regression Analysis

Linear Regression is a commonly used supervised Machine Learning algorithm

for data science learners that predicts continuous values. Linear Regression
assumes that there is a linear relationship present between dependent and
independent variables. In simple words, it finds the best-fitting line/plane that
describes two or more variables. On the other hand, Logistic Regression is
another supervised Machine Learning algorithm that helps fundamentally in
binary classification (separating discreet values).
Use of Regression Analysis
Regression analysis is primarily used for two conceptually distinct purposes.
1. Regression analysis is widely used for prediction and forecasting.
2. In certain situations, regression analysis can be used to infer causal relationships
between the independent and dependent variables.
What Is Linear Regression?
Linear Regression is a supervised learning predictive modeling algorithm in
machine learning. The model predicts value according to independent variables
and helps in finding the relationship between those variables.
Types of Regression:
1. Simple Linear Regression: Simple Linear Regression is the model that
estimates the relationship between one independent variable and one dependent
variable or target variable using a straight line.
2. Multiple Linear Regression: Multiple linear regression is a model that is used
to analyze the relationship between two or more independent variables and a
single dependent variable or target variable.
What Is Logistic Regression?
Logistic Regression is a classification algorithm, used to classify elements of a
set into two groups (binary classification) by calculating the probability of each
element of the set Logistic Regression is the appropriate regression analysis to
conduct when the dependent variable has a binary solution, we predict the
values of categorical variables.
Comparison of Linear Regression & Logistic Regression
Let us consider a problem where we are given a dataset containing the Height
and Weight of a group of people. Our task is to predict the Weight of new entries
in the Height column.
So we can figure out that this is a regression problem where we will build a
Linear Regression model. We will train the model with provided Height and
Weight values. Once the model is trained we can predict Weight for a given
unknown Height value.

Fig 1: Linear Regression

Now suppose we have an additional field Obesity and we have to classify
whether a person is obese or not depending on their provided height and weight.
This is clearly a classification problem where we have to segregate the dataset
into two classes (Obese and Not-Obese).
So, for the new problem, we can again follow the Linear Regression steps and
build a regression line. This time, the line will be based on two parameters
Height and Weight and the regression line will fit between two discreet sets of
values. As this regression line is highly susceptible to outliers, it will not do a
good job of classifying two classes.
To get a better classification, we will feed the output values from the regression
line to the sigmoid function. The sigmoid function returns the probability for each
output value from the regression line. Now based on a predefined threshold
value, we can easily classify the output into two classes Obese or Not-Obese.
Comparing Graphical Patterns: Logistic Regression Vs Linear Regression

Fig 2: Linear Regression Vs Logistic Regression

Confusion Matrix
A confusion matrix is a table that defines the performance of the classification
algorithm. It visualizes and summarizes the performance of the classification
algorithm. The most frequently used performance metrics for classification
according to these values are accuracy (ACC), precision (P), sensitivity (Sn),
specificity (Sp), and F-score values.
Overfitting
It occurs when our model tries to cover all the data points or more than the
required data points present in the given dataset.
Finally, we can summarize the similarities and differences between these two
models.

The linear and logistic probability models are given by the following equations:

From eq 1 and 2, probability (p) is considered a linear function of the regressors

for the linear model. Whereas, for the logistic model, the log odds p/(1-p) are
considered a regressors’ linear function
.
The Similarities Between Linear Regression and Logistic Regression
Linear Regression and Logistic Regression both are supervised Machine
Learning algorithms.
Linear Regression and Logistic Regression, both models are parametric
regression i.e. both models use linear equations for predictions

Differences Between Linear Regression and Logistic Regression

 Linear Regression is used to handle regression problems whereas Logistic
regression is used to handle classification problems.
 Linear regression provides a continuous output but Logistic regression
provides discreet output.
 The purpose of Linear Regression is to find the best-fitted line while
Logistic regression is one step ahead and fitting the line values to the
sigmoid curve.
 The method for calculating loss function in linear regression is the mean
squared error whereas for logistic regression it is maximum likelihood
estimation.

Linear Regression Logistic Regression
Used to predict the continuous Used to predict the categorical
dependent variable using a given dependent variable using a given
set of independent variables. set of independent variables.
The outputs produced must be a The outputs produced must be
continuous value, such as price Categorical values such as 0 or 1,
and age. Yes or No.
Conclusion
Linear Regression and Logistic Regression both are supervised Machine
Learning algorithms. Linear Regression and Logistic Regression, both models
are parametric regression i.e. both the models use linear equations for
predictions. Logistic regression is considered a generalized linear model
because the outcome depends on the sum of the inputs and parameters.