Regularization: Difference between revisions
From Rice Wiki
(Rice moved page Regularization to Lasso regression) Tag: New redirect |
(Removed redirect to Lasso regression) Tag: Removed redirect |
||
| Line 1: | Line 1: | ||
[[Category:Machine Learning]] | |||
'''Regularization''' is a technique that prevents [[overfit]]ting in machine learning. | |||
= Techniques = | |||
There are primarily two regularization techniques: | |||
* L1: [[Lasso regression]] | |||
* L2: [[Ridge regression]] | |||
The two differ slightly in degree of regularization but has similar principles: applying a penalty term to the loss function to discourage many large weights. | |||
= Regularization parameter = | |||
The '''regularization parameter''' controls the degree to which the model is regularized; a higher value penalizes large weights more. | |||
When the regularization parameter is ''too high'', some weights tend toward zero. The range of regularization parameter depends on the following | |||
* scale of the weights of the model | |||
* magnitude of noise | |||
Revision as of 19:57, 18 May 2024
Regularization is a technique that prevents overfitting in machine learning.
Techniques
There are primarily two regularization techniques:
- L1: Lasso regression
- L2: Ridge regression
The two differ slightly in degree of regularization but has similar principles: applying a penalty term to the loss function to discourage many large weights.
Regularization parameter
The regularization parameter controls the degree to which the model is regularized; a higher value penalizes large weights more.
When the regularization parameter is too high, some weights tend toward zero. The range of regularization parameter depends on the following
- scale of the weights of the model
- magnitude of noise
