Scribd
Upload
6 views

Linear Regression: Level:4 Department: IT, Security

Uploaded by

hanymqbl13
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
6 views

Linear Regression: Level:4 Department: IT, Security

Uploaded by

hanymqbl13
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 35

Lecture 2

Linear Regression

Department : IT , Security

Level :4

T. Randa Alsorori
What is machine learning?

“Machine learning is the field of study that gives computers

the ability to learn without being explicitly programmed”.

• Programming = input + Equation output

• Machine learning=input + output Equation


How does machine learning work?
• “Meta-programming”
o Predict – apply rules to examples
o Score – get feedback on performance
o Learn – change predictor to do better
How does machine learning work?
Regression vs. Classification
Linear regression with one variable

 Model Representation

 Cost function

 Gradient Descent
Model Representation(cont…)
Model Representation(cont…)
Model Representation(cont…)
Model Representation(cont…)
Linear Equations
Types Regression Models
Cost Function

 The cost function, let us figure out how to fit the best possible
straight line to our data.
Scatter Plot

1. Plot of All ( 𝑥𝑖 , 𝑦𝑖 ) 𝑝𝑎𝑖𝑟𝑠


2. Suggests How Well Model Will Fit
Thinking Challenge

How would you draw a line through the points?


How do you determine which line ‘fits best’?
Thinking Challenge

How would you draw a line through the points?


How do you determine which line ‘fits best’?
Thinking Challenge

How would you draw a line through the points?


How do you determine which line ‘fits best’?
Thinking Challenge

How would you draw a line through the points?


How do you determine which line ‘fits best’?
Least Squared errors linear Regression
Cost Function
Cost Function Visualization
Consider a simple case of hypothesis by setting 𝜽𝟎 =0, then h becomes: 𝒉𝜽 (x)=𝜽𝟏 x
Each value of 𝜽𝟏 corresponds to a different hypothesis as it is the slope of the line
which corresponds to different lines passing through the origin as shown in plots
below as y-intercept i.e. 𝜃0 is nulled out.
Cost Function Visualization

On plotting points like this further, one gets the


following graph for the cost function which is
dependent on parameter 𝜃1 .

Plot each value of 𝜃1 corresponds to a different


hypothesizes
Cost Function Visualization

What is the optimal value of 𝜽𝟏 that minimizes j(𝜽𝟏 )?


it is clear that best value for 𝜽𝟏 =1 as j(𝜽𝟏 )=0, which is
the minimum.

How to find the best value for 𝜽𝟏 ?


Plotting ?? Not practical specially in high dimensions?

The solution:
1. Analytical solution: not applicable for large
datasets
2. Numerical solution: ex:Gradient descent.
Cost Function (Recap)
Gradient Descent

 Iterative solution not only in linear regression. It’s actually used

all over the place in machine learning.

 Objective: minimize any function (Cost Function j)


Problem Setup

Have some function 𝑗(𝜃0 ,𝜃1 )


Want min 𝑗(𝜃0 ,𝜃1 )
𝜃0 ,𝜃1

Outline:
• Start with some 𝜃0 ,𝜃1
• Keep changing 𝜃0 ,𝜃1 to reduce 𝑗(𝜃0 ,𝜃1 ) until we hopefully
end up at a minimum
Uni-Variant
Multi-Variable
Gradient Descent

• “Debugging”: How to make sure gradient descent is


working correctly.
• How to choose learning rate 𝛼
Learning rate
Learning rate
Making sure gradient decent is working correctly.
Learning rate
Summary:
• If 𝛼 is too small: slow convergence.
• If 𝛼 is too large:𝑗(𝜃)may not decrease on every itration; may
not converge.

In order to choose optimum value of α run the algorithm with different


values like:

To choose 𝜶, try

…, 0.001, 0.003 , 0.01 , 0.3 , 1 ,…..

Plot the learning curve to understand whether the value should be


increased or decreased
Gradient Descent
Gradient Descent For Multiple Variable
main Reference

 Machine Learning , Ebook-2nd Ed-2020 , A Bayesian and


Optimization Perspective

You might also like