Regularization: Difference between revisions

From Rice Wiki
(Rice moved page Regularization to Lasso regression)
Tag: New redirect
 
 
(One intermediate revision by the same user not shown)
Line 1: Line 1:
#REDIRECT [[Lasso regression]]
[[Category:Machine Learning]]
 
'''Regularization''' is a technique that prevents [[overfit]]ting in machine learning.
 
= Techniques =
 
There are primarily two regularization techniques:
* L1: [[Lasso regression]]
* L2: [[Ridge regression]]
 
The two differ slightly in degree of regularization but has similar principles: applying a penalty term to the loss function to discourage many large weights.
 
There are other regularization techniques
* [[Dropout regularization]]
 
= Regularization parameter =
 
The '''regularization parameter''' controls the degree to which the model is regularized; a higher value penalizes large weights more.
 
When the regularization parameter is ''too high'', some weights tend toward zero. The range of regularization parameter depends on the following
* scale of the weights of the model
* magnitude of noise

Latest revision as of 20:20, 18 May 2024


Regularization is a technique that prevents overfitting in machine learning.

Techniques

There are primarily two regularization techniques:

The two differ slightly in degree of regularization but has similar principles: applying a penalty term to the loss function to discourage many large weights.

There are other regularization techniques

Regularization parameter

The regularization parameter controls the degree to which the model is regularized; a higher value penalizes large weights more.

When the regularization parameter is too high, some weights tend toward zero. The range of regularization parameter depends on the following

  • scale of the weights of the model
  • magnitude of noise