Gradient Descent: Difference between revisions
From Rice Wiki
No edit summary |
No edit summary |
||
| (One intermediate revision by the same user not shown) | |||
| Line 1: | Line 1: | ||
'''Gradient descent (GD)''' is an iterative method to update the weights of a | '''Gradient descent (GD)''' is an iterative method to update the weights of a [[Linear Regression]] model to achieve the optimal model. | ||
The downside to GD is that It does not gaurantee the global optimal. To achieve global minimum, tradeoff needs to be made. | The downside to GD is that It does not gaurantee the global optimal. To achieve global minimum, tradeoff needs to be made. | ||
| Line 17: | Line 17: | ||
* [[Stochastic Gradient Descent]] | * [[Stochastic Gradient Descent]] | ||
* [[Batch Gradient Descent]] | * [[Batch Gradient Descent]] | ||
[[Category:Machine Learning]] | |||
Latest revision as of 19:33, 17 May 2024
Gradient descent (GD) is an iterative method to update the weights of a Linear Regression model to achieve the optimal model.
The downside to GD is that It does not gaurantee the global optimal. To achieve global minimum, tradeoff needs to be made.
Epoch
1 epoch is completed when the entire instances in the training sets are processed once to update the weights of the model.
Hyperparameters
The learning rate is a hyperparameter of gradient descent
LMS
Least-mean-squared is a GD update rule.
Optimizations
There are two optimization of gradient descents:
