Scribd
Upload
13 views

Gradient Descent

Uploaded by

sankeerthmanmadhan2002
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
13 views

Gradient Descent

Uploaded by

sankeerthmanmadhan2002
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 12

What is Gradient Descent?

Gradient descent is an optimization algorithm which is commonly-used to train


machine learning models and neural networks. Training data helps these models
learn over time, and the cost function within gradient descent specifically acts as a barometer,
gauging its accuracy with each iteration of parameter updates. Until the function is close to or
equal to zero, the model will continue to adjust its parameters to yield the smallest possible
error. Once machine learning models are optimized for accuracy, they can be powerful tools
for artificial intelligence (AI) and computer science applications.
You are playing a game where the players are at the top of a mountain, and they are asked to
reach the lowest point of the mountain.

Additionally, they are blindfolded. So, what approach do you think would make you reach
the lake?

The best way is to observe the ground and find where the land descends.

From that position, take a step in the descending direction and iterate this process until we
reach the lowest point.
Plotting a scatterplot in statistics and ‰finding the line of best fit, which required calculating
the error between the actual output and the predicted output (y-hat) using the mean squared
error formula.

The gradient descent algorithm behaves similarly, but it is based on a convex function, such
as the one below:
High learning rates result in larger steps but risks overshooting the minimum.

Conversely, a low learning rate has small step sizes.

While it has the advantage of more precision, the number of iterations compromises overall
efficiency as this takes more time and computations to reach the minimum.

You might also like