Maximum likelihood estimation: Difference between revisions
Line 15: | Line 15: | ||
<math>L(w_i, \sigma^2|x,y) = \prod \frac{1}{ \sqrt{ 2 \pi \sigma^2}} exp \left( - \frac{(y_i - g(x_i))^2}{2 \sigma^2 } \right)</math> | <math>L(w_i, \sigma^2|x,y) = \prod \frac{1}{ \sqrt{ 2 \pi \sigma^2}} exp \left( - \frac{(y_i - g(x_i))^2}{2 \sigma^2 } \right)</math> | ||
The likelihood of observing the data is the product of observing each data point, given by the probability density function of standard distribution. | |||
The weights are then changed to fit it better, and the process repeats. | The weights are then changed to fit it better, and the process repeats. |
Revision as of 04:09, 25 April 2024
Maximum likelihood estimation (MLE) is one of the methods to find the coefficients of a model that minimizes the RSS in linear regression. MLE does this by maximizing the likelihood of observing the training data given a model.
Background
Consider objective function
where is the true relationship and is the residual error/noise
We assume that , and
Likelihood function
The likelihood function determines the likelihood of observing the data given the parameters of the model. A high likelihood indicates a good model.
The likelihood of observing the data is the product of observing each data point, given by the probability density function of standard distribution.
The weights are then changed to fit it better, and the process repeats.
The computation can be simplified to the following
<math