SlideShare a Scribd company logo
Linear Regression
with gradient
Descent
A guide by landofai.com
Code: Linear regression
Gradient Descent
Gradient Descent is the most popular optimization
strategy, used machine learning and deep learning
right now.
It can be combined with almost every algorithm
yet it is easy to understand. So, everyone planning
to go on the journey of machine learning should
understand this.
Intuition
It is simply used to find the values of the
parameters at which the given function reaches
its nearest minimum cost.
Intuition
"A gradient is the ratio which relates the input and output of a
function. How small changes drives changes in the output of the
function."
Suppose we have a function f(x) = x2. Then the derivative of the
function, f’(x) is 2*x. It means if the x is changed 1 unit then f(x) will
change 2*1.
1. A blindfolded person starts at top of hill.
2. Checks for the steepest direction
downward on that point.
3. Take a step in that direction.
4. Again checks for the steepest direction
towards downward.
5. Repeat until the steep/gradient is
acceptable or is flat.
Math proving this
The equation below shows how it's done: 'x(next)' is the new
position of the person, 'x(current)' is the current position,
subtraction means that we 'gamma is the step and 'nabla f(x)'
is the gradient showing the steepest direction.
Let's take another example
of Linear regression
technique in machine
learning,
We have to find optimal 'w'
and 'b' for the cost function
J(w, b) where J is minimum.
below is an illustration of
convex function, (w and b)
are represented on the
horizontal axes, while J(w, b)
is represented on the vertical
axis.
Learning rate
The steps which are taken to reach optimal point decides the rate
of gradient descent. It is often referred to as 'Learning rate'(i.e.,
The size of the steps).
➔ Too big
bounce between the convex function and may not reach
the local minimum.
➔ Too small
gradient descent will eventually reach the local minimum
but it will take too much time for that
➔ Just right
gradient descent will eventually reach the local minimum
but it will take too much time for that
Linear regression with gradient descent
Gradient Descent types
● Batch Gradient Descent
A.k.a. Vanilla Gradient Descent. Calculates error for each
example. Model is updated only after an epoch.
● Stochastic Gradient descent
SGD unlike vanilla, iterates over each example while updating
the model. Frequent updates can be computationally more
expensive.
● Mini Batch Gradient Descent
a combination of concepts of both SGD and Batch Gradient
Descent.
○ Splits data into batches then performs update on
batches balancing between the efficiency of batch
gradient descent and the robustness of SGD.
Linear Regression.
Just Give me The code.: GradientDescentDemo
Visitors to his store, mostly tourists, speak many
different languages making anything beyond a simple
transaction a challenge.
Y=mX+b
1. Our goal is to best fit a line for given
points.
2. Start by random m and b.
3. Calculate error between predicted Y and
true Y.
4. Adjust m and b with gradient descent
5. Repeat until satisfactory result is achieved.
Error
Here, Mean Squared Error(MSE)
How to update m and b
Updated value = old value - learning rare * gradient
Good luck!
Thank You for your interest.
AI Adventures
Ad

More Related Content

What's hot (20)

Machine Learning With Logistic Regression
Machine Learning  With Logistic RegressionMachine Learning  With Logistic Regression
Machine Learning With Logistic Regression
Knoldus Inc.
 
Logistic regression in Machine Learning
Logistic regression in Machine LearningLogistic regression in Machine Learning
Logistic regression in Machine Learning
Kuppusamy P
 
Dimensionality Reduction
Dimensionality ReductionDimensionality Reduction
Dimensionality Reduction
mrizwan969
 
K Nearest Neighbors
K Nearest NeighborsK Nearest Neighbors
K Nearest Neighbors
Tilani Gunawardena PhD(UNIBAS), BSc(Pera), FHEA(UK), CEng, MIESL
 
Linear Discriminant Analysis (LDA)
Linear Discriminant Analysis (LDA)Linear Discriminant Analysis (LDA)
Linear Discriminant Analysis (LDA)
Anmol Dwivedi
 
Classification Based Machine Learning Algorithms
Classification Based Machine Learning AlgorithmsClassification Based Machine Learning Algorithms
Classification Based Machine Learning Algorithms
Md. Main Uddin Rony
 
An overview of gradient descent optimization algorithms
An overview of gradient descent optimization algorithms An overview of gradient descent optimization algorithms
An overview of gradient descent optimization algorithms
Hakky St
 
Bias and variance trade off
Bias and variance trade offBias and variance trade off
Bias and variance trade off
VARUN KUMAR
 
Gradient descent method
Gradient descent methodGradient descent method
Gradient descent method
Sanghyuk Chun
 
Overfitting & Underfitting
Overfitting & UnderfittingOverfitting & Underfitting
Overfitting & Underfitting
SOUMIT KAR
 
Presentation on K-Means Clustering
Presentation on K-Means ClusteringPresentation on K-Means Clustering
Presentation on K-Means Clustering
Pabna University of Science & Technology
 
An Introduction to Supervised Machine Learning and Pattern Classification: Th...
An Introduction to Supervised Machine Learning and Pattern Classification: Th...An Introduction to Supervised Machine Learning and Pattern Classification: Th...
An Introduction to Supervised Machine Learning and Pattern Classification: Th...
Sebastian Raschka
 
Support Vector Machines ( SVM )
Support Vector Machines ( SVM ) Support Vector Machines ( SVM )
Support Vector Machines ( SVM )
Mohammad Junaid Khan
 
MACHINE LEARNING - GENETIC ALGORITHM
MACHINE LEARNING - GENETIC ALGORITHMMACHINE LEARNING - GENETIC ALGORITHM
MACHINE LEARNING - GENETIC ALGORITHM
Puneet Kulyana
 
Optimization/Gradient Descent
Optimization/Gradient DescentOptimization/Gradient Descent
Optimization/Gradient Descent
kandelin
 
Optimization in Deep Learning
Optimization in Deep LearningOptimization in Deep Learning
Optimization in Deep Learning
Yan Xu
 
Machine Learning with Decision trees
Machine Learning with Decision treesMachine Learning with Decision trees
Machine Learning with Decision trees
Knoldus Inc.
 
Ensemble learning
Ensemble learningEnsemble learning
Ensemble learning
Haris Jamil
 
Class imbalance problem1
Class imbalance problem1Class imbalance problem1
Class imbalance problem1
chs71
 
Random forest
Random forestRandom forest
Random forest
Ujjawal
 
Machine Learning With Logistic Regression
Machine Learning  With Logistic RegressionMachine Learning  With Logistic Regression
Machine Learning With Logistic Regression
Knoldus Inc.
 
Logistic regression in Machine Learning
Logistic regression in Machine LearningLogistic regression in Machine Learning
Logistic regression in Machine Learning
Kuppusamy P
 
Dimensionality Reduction
Dimensionality ReductionDimensionality Reduction
Dimensionality Reduction
mrizwan969
 
Linear Discriminant Analysis (LDA)
Linear Discriminant Analysis (LDA)Linear Discriminant Analysis (LDA)
Linear Discriminant Analysis (LDA)
Anmol Dwivedi
 
Classification Based Machine Learning Algorithms
Classification Based Machine Learning AlgorithmsClassification Based Machine Learning Algorithms
Classification Based Machine Learning Algorithms
Md. Main Uddin Rony
 
An overview of gradient descent optimization algorithms
An overview of gradient descent optimization algorithms An overview of gradient descent optimization algorithms
An overview of gradient descent optimization algorithms
Hakky St
 
Bias and variance trade off
Bias and variance trade offBias and variance trade off
Bias and variance trade off
VARUN KUMAR
 
Gradient descent method
Gradient descent methodGradient descent method
Gradient descent method
Sanghyuk Chun
 
Overfitting & Underfitting
Overfitting & UnderfittingOverfitting & Underfitting
Overfitting & Underfitting
SOUMIT KAR
 
An Introduction to Supervised Machine Learning and Pattern Classification: Th...
An Introduction to Supervised Machine Learning and Pattern Classification: Th...An Introduction to Supervised Machine Learning and Pattern Classification: Th...
An Introduction to Supervised Machine Learning and Pattern Classification: Th...
Sebastian Raschka
 
MACHINE LEARNING - GENETIC ALGORITHM
MACHINE LEARNING - GENETIC ALGORITHMMACHINE LEARNING - GENETIC ALGORITHM
MACHINE LEARNING - GENETIC ALGORITHM
Puneet Kulyana
 
Optimization/Gradient Descent
Optimization/Gradient DescentOptimization/Gradient Descent
Optimization/Gradient Descent
kandelin
 
Optimization in Deep Learning
Optimization in Deep LearningOptimization in Deep Learning
Optimization in Deep Learning
Yan Xu
 
Machine Learning with Decision trees
Machine Learning with Decision treesMachine Learning with Decision trees
Machine Learning with Decision trees
Knoldus Inc.
 
Ensemble learning
Ensemble learningEnsemble learning
Ensemble learning
Haris Jamil
 
Class imbalance problem1
Class imbalance problem1Class imbalance problem1
Class imbalance problem1
chs71
 
Random forest
Random forestRandom forest
Random forest
Ujjawal
 

Similar to Linear regression with gradient descent (20)

4. OPTIMIZATION NN AND FL.pptx
4. OPTIMIZATION NN AND FL.pptx4. OPTIMIZATION NN AND FL.pptx
4. OPTIMIZATION NN AND FL.pptx
kumarkaushal17
 
Gradient Descent DS Rohit Sharma fench knjs.pptx
Gradient Descent DS Rohit Sharma fench knjs.pptxGradient Descent DS Rohit Sharma fench knjs.pptx
Gradient Descent DS Rohit Sharma fench knjs.pptx
businessmarketing100
 
Steepest descent method in sc
Steepest descent method in scSteepest descent method in sc
Steepest descent method in sc
rajshreemuthiah
 
Deep learning concepts
Deep learning conceptsDeep learning concepts
Deep learning concepts
Joe li
 
Steepest descent method
Steepest descent methodSteepest descent method
Steepest descent method
Prof. Neeta Awasthy
 
An overview of gradient descent optimization algorithms.pdf
An overview of gradient descent optimization algorithms.pdfAn overview of gradient descent optimization algorithms.pdf
An overview of gradient descent optimization algorithms.pdf
vudinhphuong96
 
4 linear regeression with multiple variables
4 linear regeression with multiple variables4 linear regeression with multiple variables
4 linear regeression with multiple variables
TanmayVijay1
 
MACHINE LEARNING NEURAL NETWORK PPT UNIT 4
MACHINE LEARNING NEURAL NETWORK PPT UNIT 4MACHINE LEARNING NEURAL NETWORK PPT UNIT 4
MACHINE LEARNING NEURAL NETWORK PPT UNIT 4
MulliMary
 
Deep Learning for Computer Vision: Optimization (UPC 2016)
Deep Learning for Computer Vision: Optimization (UPC 2016)Deep Learning for Computer Vision: Optimization (UPC 2016)
Deep Learning for Computer Vision: Optimization (UPC 2016)
Universitat Politècnica de Catalunya
 
Auto encoders in Deep Learning
Auto encoders in Deep LearningAuto encoders in Deep Learning
Auto encoders in Deep Learning
Shajun Nisha
 
Dep Neural Networks introduction new.pdf
Dep Neural Networks introduction new.pdfDep Neural Networks introduction new.pdf
Dep Neural Networks introduction new.pdf
ratnababum
 
Deep learning architectures
Deep learning architecturesDeep learning architectures
Deep learning architectures
Joe li
 
Gradient Descent or Assent is to find optimal parameters that minimize the l...
Gradient Descent or Assent  is to find optimal parameters that minimize the l...Gradient Descent or Assent  is to find optimal parameters that minimize the l...
Gradient Descent or Assent is to find optimal parameters that minimize the l...
MakalaRamesh1
 
2. Linear regression with one variable.pptx
2. Linear regression with one variable.pptx2. Linear regression with one variable.pptx
2. Linear regression with one variable.pptx
Emad Nabil
 
Methods of Optimization in Machine Learning
Methods of Optimization in Machine LearningMethods of Optimization in Machine Learning
Methods of Optimization in Machine Learning
Knoldus Inc.
 
Deep Neural Network Module 3A Optimization.pptx
Deep Neural Network Module 3A Optimization.pptxDeep Neural Network Module 3A Optimization.pptx
Deep Neural Network Module 3A Optimization.pptx
ratnababum
 
Advance Machine Learning presentation.pptx
Advance Machine Learning presentation.pptxAdvance Machine Learning presentation.pptx
Advance Machine Learning presentation.pptx
ImXaib
 
Application of Derivative Class 12th Best Project by Shubham prasad
Application of Derivative Class 12th Best Project by Shubham prasadApplication of Derivative Class 12th Best Project by Shubham prasad
Application of Derivative Class 12th Best Project by Shubham prasad
Shubham Prasad
 
Linear logisticregression
Linear logisticregressionLinear logisticregression
Linear logisticregression
kongara
 
Master of Canvas
Master of CanvasMaster of Canvas
Master of Canvas
Mima Yuki
 
4. OPTIMIZATION NN AND FL.pptx
4. OPTIMIZATION NN AND FL.pptx4. OPTIMIZATION NN AND FL.pptx
4. OPTIMIZATION NN AND FL.pptx
kumarkaushal17
 
Gradient Descent DS Rohit Sharma fench knjs.pptx
Gradient Descent DS Rohit Sharma fench knjs.pptxGradient Descent DS Rohit Sharma fench knjs.pptx
Gradient Descent DS Rohit Sharma fench knjs.pptx
businessmarketing100
 
Steepest descent method in sc
Steepest descent method in scSteepest descent method in sc
Steepest descent method in sc
rajshreemuthiah
 
Deep learning concepts
Deep learning conceptsDeep learning concepts
Deep learning concepts
Joe li
 
An overview of gradient descent optimization algorithms.pdf
An overview of gradient descent optimization algorithms.pdfAn overview of gradient descent optimization algorithms.pdf
An overview of gradient descent optimization algorithms.pdf
vudinhphuong96
 
4 linear regeression with multiple variables
4 linear regeression with multiple variables4 linear regeression with multiple variables
4 linear regeression with multiple variables
TanmayVijay1
 
MACHINE LEARNING NEURAL NETWORK PPT UNIT 4
MACHINE LEARNING NEURAL NETWORK PPT UNIT 4MACHINE LEARNING NEURAL NETWORK PPT UNIT 4
MACHINE LEARNING NEURAL NETWORK PPT UNIT 4
MulliMary
 
Auto encoders in Deep Learning
Auto encoders in Deep LearningAuto encoders in Deep Learning
Auto encoders in Deep Learning
Shajun Nisha
 
Dep Neural Networks introduction new.pdf
Dep Neural Networks introduction new.pdfDep Neural Networks introduction new.pdf
Dep Neural Networks introduction new.pdf
ratnababum
 
Deep learning architectures
Deep learning architecturesDeep learning architectures
Deep learning architectures
Joe li
 
Gradient Descent or Assent is to find optimal parameters that minimize the l...
Gradient Descent or Assent  is to find optimal parameters that minimize the l...Gradient Descent or Assent  is to find optimal parameters that minimize the l...
Gradient Descent or Assent is to find optimal parameters that minimize the l...
MakalaRamesh1
 
2. Linear regression with one variable.pptx
2. Linear regression with one variable.pptx2. Linear regression with one variable.pptx
2. Linear regression with one variable.pptx
Emad Nabil
 
Methods of Optimization in Machine Learning
Methods of Optimization in Machine LearningMethods of Optimization in Machine Learning
Methods of Optimization in Machine Learning
Knoldus Inc.
 
Deep Neural Network Module 3A Optimization.pptx
Deep Neural Network Module 3A Optimization.pptxDeep Neural Network Module 3A Optimization.pptx
Deep Neural Network Module 3A Optimization.pptx
ratnababum
 
Advance Machine Learning presentation.pptx
Advance Machine Learning presentation.pptxAdvance Machine Learning presentation.pptx
Advance Machine Learning presentation.pptx
ImXaib
 
Application of Derivative Class 12th Best Project by Shubham prasad
Application of Derivative Class 12th Best Project by Shubham prasadApplication of Derivative Class 12th Best Project by Shubham prasad
Application of Derivative Class 12th Best Project by Shubham prasad
Shubham Prasad
 
Linear logisticregression
Linear logisticregressionLinear logisticregression
Linear logisticregression
kongara
 
Master of Canvas
Master of CanvasMaster of Canvas
Master of Canvas
Mima Yuki
 
Ad

Recently uploaded (20)

New Microsoft PowerPoint Presentation.pptx
New Microsoft PowerPoint Presentation.pptxNew Microsoft PowerPoint Presentation.pptx
New Microsoft PowerPoint Presentation.pptx
milanasargsyan5
 
Exploring-Substances-Acidic-Basic-and-Neutral.pdf
Exploring-Substances-Acidic-Basic-and-Neutral.pdfExploring-Substances-Acidic-Basic-and-Neutral.pdf
Exploring-Substances-Acidic-Basic-and-Neutral.pdf
Sandeep Swamy
 
Phoenix – A Collaborative Renewal of Children’s and Young People’s Services C...
Phoenix – A Collaborative Renewal of Children’s and Young People’s Services C...Phoenix – A Collaborative Renewal of Children’s and Young People’s Services C...
Phoenix – A Collaborative Renewal of Children’s and Young People’s Services C...
Library Association of Ireland
 
SPRING FESTIVITIES - UK AND USA -
SPRING FESTIVITIES - UK AND USA            -SPRING FESTIVITIES - UK AND USA            -
SPRING FESTIVITIES - UK AND USA -
Colégio Santa Teresinha
 
YSPH VMOC Special Report - Measles Outbreak Southwest US 4-30-2025.pptx
YSPH VMOC Special Report - Measles Outbreak  Southwest US 4-30-2025.pptxYSPH VMOC Special Report - Measles Outbreak  Southwest US 4-30-2025.pptx
YSPH VMOC Special Report - Measles Outbreak Southwest US 4-30-2025.pptx
Yale School of Public Health - The Virtual Medical Operations Center (VMOC)
 
One Hot encoding a revolution in Machine learning
One Hot encoding a revolution in Machine learningOne Hot encoding a revolution in Machine learning
One Hot encoding a revolution in Machine learning
momer9505
 
YSPH VMOC Special Report - Measles Outbreak Southwest US 4-26-2025.pptx
YSPH VMOC Special Report - Measles Outbreak  Southwest US 4-26-2025.pptxYSPH VMOC Special Report - Measles Outbreak  Southwest US 4-26-2025.pptx
YSPH VMOC Special Report - Measles Outbreak Southwest US 4-26-2025.pptx
Yale School of Public Health - The Virtual Medical Operations Center (VMOC)
 
Marie Boran Special Collections Librarian Hardiman Library, University of Gal...
Marie Boran Special Collections Librarian Hardiman Library, University of Gal...Marie Boran Special Collections Librarian Hardiman Library, University of Gal...
Marie Boran Special Collections Librarian Hardiman Library, University of Gal...
Library Association of Ireland
 
CBSE - Grade 8 - Science - Chemistry - Metals and Non Metals - Worksheet
CBSE - Grade 8 - Science - Chemistry - Metals and Non Metals - WorksheetCBSE - Grade 8 - Science - Chemistry - Metals and Non Metals - Worksheet
CBSE - Grade 8 - Science - Chemistry - Metals and Non Metals - Worksheet
Sritoma Majumder
 
UNIT 3 NATIONAL HEALTH PROGRAMMEE. SOCIAL AND PREVENTIVE PHARMACY
UNIT 3 NATIONAL HEALTH PROGRAMMEE. SOCIAL AND PREVENTIVE PHARMACYUNIT 3 NATIONAL HEALTH PROGRAMMEE. SOCIAL AND PREVENTIVE PHARMACY
UNIT 3 NATIONAL HEALTH PROGRAMMEE. SOCIAL AND PREVENTIVE PHARMACY
DR.PRISCILLA MARY J
 
Ultimate VMware 2V0-11.25 Exam Dumps for Exam Success
Ultimate VMware 2V0-11.25 Exam Dumps for Exam SuccessUltimate VMware 2V0-11.25 Exam Dumps for Exam Success
Ultimate VMware 2V0-11.25 Exam Dumps for Exam Success
Mark Soia
 
Presentation on Tourism Product Development By Md Shaifullar Rabbi
Presentation on Tourism Product Development By Md Shaifullar RabbiPresentation on Tourism Product Development By Md Shaifullar Rabbi
Presentation on Tourism Product Development By Md Shaifullar Rabbi
Md Shaifullar Rabbi
 
How to track Cost and Revenue using Analytic Accounts in odoo Accounting, App...
How to track Cost and Revenue using Analytic Accounts in odoo Accounting, App...How to track Cost and Revenue using Analytic Accounts in odoo Accounting, App...
How to track Cost and Revenue using Analytic Accounts in odoo Accounting, App...
Celine George
 
Presentation of the MIPLM subject matter expert Erdem Kaya
Presentation of the MIPLM subject matter expert Erdem KayaPresentation of the MIPLM subject matter expert Erdem Kaya
Presentation of the MIPLM subject matter expert Erdem Kaya
MIPLM
 
Handling Multiple Choice Responses: Fortune Effiong.pptx
Handling Multiple Choice Responses: Fortune Effiong.pptxHandling Multiple Choice Responses: Fortune Effiong.pptx
Handling Multiple Choice Responses: Fortune Effiong.pptx
AuthorAIDNationalRes
 
How to manage Multiple Warehouses for multiple floors in odoo point of sale
How to manage Multiple Warehouses for multiple floors in odoo point of saleHow to manage Multiple Warehouses for multiple floors in odoo point of sale
How to manage Multiple Warehouses for multiple floors in odoo point of sale
Celine George
 
The ever evoilving world of science /7th class science curiosity /samyans aca...
The ever evoilving world of science /7th class science curiosity /samyans aca...The ever evoilving world of science /7th class science curiosity /samyans aca...
The ever evoilving world of science /7th class science curiosity /samyans aca...
Sandeep Swamy
 
How to Customize Your Financial Reports & Tax Reports With Odoo 17 Accounting
How to Customize Your Financial Reports & Tax Reports With Odoo 17 AccountingHow to Customize Your Financial Reports & Tax Reports With Odoo 17 Accounting
How to Customize Your Financial Reports & Tax Reports With Odoo 17 Accounting
Celine George
 
Open Access: Revamping Library Learning Resources.
Open Access: Revamping Library Learning Resources.Open Access: Revamping Library Learning Resources.
Open Access: Revamping Library Learning Resources.
Rishi Bankim Chandra Evening College, Naihati, North 24 Parganas, West Bengal, India
 
High Performance Liquid Chromatography .pptx
High Performance Liquid Chromatography .pptxHigh Performance Liquid Chromatography .pptx
High Performance Liquid Chromatography .pptx
Ayush Srivastava
 
New Microsoft PowerPoint Presentation.pptx
New Microsoft PowerPoint Presentation.pptxNew Microsoft PowerPoint Presentation.pptx
New Microsoft PowerPoint Presentation.pptx
milanasargsyan5
 
Exploring-Substances-Acidic-Basic-and-Neutral.pdf
Exploring-Substances-Acidic-Basic-and-Neutral.pdfExploring-Substances-Acidic-Basic-and-Neutral.pdf
Exploring-Substances-Acidic-Basic-and-Neutral.pdf
Sandeep Swamy
 
Phoenix – A Collaborative Renewal of Children’s and Young People’s Services C...
Phoenix – A Collaborative Renewal of Children’s and Young People’s Services C...Phoenix – A Collaborative Renewal of Children’s and Young People’s Services C...
Phoenix – A Collaborative Renewal of Children’s and Young People’s Services C...
Library Association of Ireland
 
One Hot encoding a revolution in Machine learning
One Hot encoding a revolution in Machine learningOne Hot encoding a revolution in Machine learning
One Hot encoding a revolution in Machine learning
momer9505
 
Marie Boran Special Collections Librarian Hardiman Library, University of Gal...
Marie Boran Special Collections Librarian Hardiman Library, University of Gal...Marie Boran Special Collections Librarian Hardiman Library, University of Gal...
Marie Boran Special Collections Librarian Hardiman Library, University of Gal...
Library Association of Ireland
 
CBSE - Grade 8 - Science - Chemistry - Metals and Non Metals - Worksheet
CBSE - Grade 8 - Science - Chemistry - Metals and Non Metals - WorksheetCBSE - Grade 8 - Science - Chemistry - Metals and Non Metals - Worksheet
CBSE - Grade 8 - Science - Chemistry - Metals and Non Metals - Worksheet
Sritoma Majumder
 
UNIT 3 NATIONAL HEALTH PROGRAMMEE. SOCIAL AND PREVENTIVE PHARMACY
UNIT 3 NATIONAL HEALTH PROGRAMMEE. SOCIAL AND PREVENTIVE PHARMACYUNIT 3 NATIONAL HEALTH PROGRAMMEE. SOCIAL AND PREVENTIVE PHARMACY
UNIT 3 NATIONAL HEALTH PROGRAMMEE. SOCIAL AND PREVENTIVE PHARMACY
DR.PRISCILLA MARY J
 
Ultimate VMware 2V0-11.25 Exam Dumps for Exam Success
Ultimate VMware 2V0-11.25 Exam Dumps for Exam SuccessUltimate VMware 2V0-11.25 Exam Dumps for Exam Success
Ultimate VMware 2V0-11.25 Exam Dumps for Exam Success
Mark Soia
 
Presentation on Tourism Product Development By Md Shaifullar Rabbi
Presentation on Tourism Product Development By Md Shaifullar RabbiPresentation on Tourism Product Development By Md Shaifullar Rabbi
Presentation on Tourism Product Development By Md Shaifullar Rabbi
Md Shaifullar Rabbi
 
How to track Cost and Revenue using Analytic Accounts in odoo Accounting, App...
How to track Cost and Revenue using Analytic Accounts in odoo Accounting, App...How to track Cost and Revenue using Analytic Accounts in odoo Accounting, App...
How to track Cost and Revenue using Analytic Accounts in odoo Accounting, App...
Celine George
 
Presentation of the MIPLM subject matter expert Erdem Kaya
Presentation of the MIPLM subject matter expert Erdem KayaPresentation of the MIPLM subject matter expert Erdem Kaya
Presentation of the MIPLM subject matter expert Erdem Kaya
MIPLM
 
Handling Multiple Choice Responses: Fortune Effiong.pptx
Handling Multiple Choice Responses: Fortune Effiong.pptxHandling Multiple Choice Responses: Fortune Effiong.pptx
Handling Multiple Choice Responses: Fortune Effiong.pptx
AuthorAIDNationalRes
 
How to manage Multiple Warehouses for multiple floors in odoo point of sale
How to manage Multiple Warehouses for multiple floors in odoo point of saleHow to manage Multiple Warehouses for multiple floors in odoo point of sale
How to manage Multiple Warehouses for multiple floors in odoo point of sale
Celine George
 
The ever evoilving world of science /7th class science curiosity /samyans aca...
The ever evoilving world of science /7th class science curiosity /samyans aca...The ever evoilving world of science /7th class science curiosity /samyans aca...
The ever evoilving world of science /7th class science curiosity /samyans aca...
Sandeep Swamy
 
How to Customize Your Financial Reports & Tax Reports With Odoo 17 Accounting
How to Customize Your Financial Reports & Tax Reports With Odoo 17 AccountingHow to Customize Your Financial Reports & Tax Reports With Odoo 17 Accounting
How to Customize Your Financial Reports & Tax Reports With Odoo 17 Accounting
Celine George
 
High Performance Liquid Chromatography .pptx
High Performance Liquid Chromatography .pptxHigh Performance Liquid Chromatography .pptx
High Performance Liquid Chromatography .pptx
Ayush Srivastava
 
Ad

Linear regression with gradient descent

  • 1. Linear Regression with gradient Descent A guide by landofai.com Code: Linear regression
  • 2. Gradient Descent Gradient Descent is the most popular optimization strategy, used machine learning and deep learning right now. It can be combined with almost every algorithm yet it is easy to understand. So, everyone planning to go on the journey of machine learning should understand this.
  • 3. Intuition It is simply used to find the values of the parameters at which the given function reaches its nearest minimum cost.
  • 4. Intuition "A gradient is the ratio which relates the input and output of a function. How small changes drives changes in the output of the function." Suppose we have a function f(x) = x2. Then the derivative of the function, f’(x) is 2*x. It means if the x is changed 1 unit then f(x) will change 2*1.
  • 5. 1. A blindfolded person starts at top of hill. 2. Checks for the steepest direction downward on that point. 3. Take a step in that direction. 4. Again checks for the steepest direction towards downward. 5. Repeat until the steep/gradient is acceptable or is flat.
  • 6. Math proving this The equation below shows how it's done: 'x(next)' is the new position of the person, 'x(current)' is the current position, subtraction means that we 'gamma is the step and 'nabla f(x)' is the gradient showing the steepest direction.
  • 7. Let's take another example of Linear regression technique in machine learning, We have to find optimal 'w' and 'b' for the cost function J(w, b) where J is minimum. below is an illustration of convex function, (w and b) are represented on the horizontal axes, while J(w, b) is represented on the vertical axis.
  • 8. Learning rate The steps which are taken to reach optimal point decides the rate of gradient descent. It is often referred to as 'Learning rate'(i.e., The size of the steps). ➔ Too big bounce between the convex function and may not reach the local minimum. ➔ Too small gradient descent will eventually reach the local minimum but it will take too much time for that ➔ Just right gradient descent will eventually reach the local minimum but it will take too much time for that
  • 10. Gradient Descent types ● Batch Gradient Descent A.k.a. Vanilla Gradient Descent. Calculates error for each example. Model is updated only after an epoch. ● Stochastic Gradient descent SGD unlike vanilla, iterates over each example while updating the model. Frequent updates can be computationally more expensive. ● Mini Batch Gradient Descent a combination of concepts of both SGD and Batch Gradient Descent. ○ Splits data into batches then performs update on batches balancing between the efficiency of batch gradient descent and the robustness of SGD.
  • 11. Linear Regression. Just Give me The code.: GradientDescentDemo Visitors to his store, mostly tourists, speak many different languages making anything beyond a simple transaction a challenge.
  • 12. Y=mX+b 1. Our goal is to best fit a line for given points. 2. Start by random m and b. 3. Calculate error between predicted Y and true Y. 4. Adjust m and b with gradient descent 5. Repeat until satisfactory result is achieved.
  • 14. How to update m and b Updated value = old value - learning rare * gradient
  • 15. Good luck! Thank You for your interest. AI Adventures