Bivariate

Consider two numerica random variables $X$ and $Y$. We can measure their covariance.

$Cov(X,Y)$

The correlation of two random variables measures the line dependent between $X$ and $Y$

$Cor(X,Y)=\rho ={\frac {Cov(X,Y)}{sd(X)sd(Y)}}$

Correlation is always between -1 and 1

Bivariate Normal

The bivariate normal (aka. bivariate gaussian) is one special type of continuous random variable.

$(X,Y)$ is bivariate normal if

1. The marginal PDF of both X and Y are normal
2. For any $x$, the condition PDF of $Y$ given $X=x$ is Normal

• • Works the other way around: Bivariate gaussian means that condition is satisfied

Predicting Y given X

Given bivariate normal, we can predict one variable given another. Let us try estimating the expected Y given X is x

$E(Y|X=x)$

There are three main methods

• Scatter plot approximation
• Joint PDF
• 5 statistics

5 Parameters

We need to know 5 parameters about $X$ and $Y$

$E(X),sd(X),E(Y),sd(Y),\rho$

If $X,Y$ follows bivariate normal distribution, then we have

$\left({\frac {E(Y|X=x)-E(Y)}{sd(Y)}}\right)=\rho \left({\frac {x-E(X)}{sd(X)}}\right)$

The left side is the predicted Z-score for Y, and the right side is the product of correlation and Z-score of X = x

The variance is given by

$Var(Y|X=x)=(1-\rho ^{2})Var(Y)$

Due to the range of $\rho$, the variance of Y given X is always smaller than the actual variance. The standard deviation is just rooted that.

Regression Effect

The regression effect is the phenomenon that the best prediction of $Y$ given $X=x$ is less rare for $Y$ than $x$; Future predictions regress to mediocrity.

When you plot all the predicted $E(Y|X=x)$, you get the linear regression line. The regression effect can be demonstrated by also plotting the SD line (where the correlation is not applied).