Perceptron learning

Figure 1. Graphical representation of linearly separable data

Perceptron learning is an algorithm for binary classification. It is good for classifying linearly separable data, as seen in figure 1. If the data is not linearly separable, more complex algorithms need to be used.

How it works

Initially, all attributes are passed into a neuron, each with a randomized weight.

$z={\bf {{w}{\bf {x}}}}$

The binary step activation function takes in z and outputs 1 or 0 depending on a threshold.

The error is computed

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle E = \frac{1}{2} \sum (y^{(i)} - \hat{y}^{(i)})^2}

where the 1/2 is for simplicity of computation after the gradient. We use the gradient of the error function to update the weights of the model

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \Delta w_j = \nabla_{w_j} E = \sum (y^{(i)} - \hat{y^{(i)}})(-x_j)}