University AI - Gaussian Distribution

Gaussian Distribution

Gaussian or Normal Distributions:

$$p(x)=\frac{1}{\sigma \sqrt{2\pi }}\exp \left\{-\frac{1}{2}{\left(\frac{x-\mu }{\sigma }\right)}^2\right\}$$

Where:

• $\mu$ : mean of the distribution
• $\sigma$ : variance of the distribution

Multivariate Gaussian Distributions:

$$p(x)=\frac{1}{|\Sigma {|}^{1/2}{\left(2\pi \right)}^{d/2}}\exp \left\{-\frac{1}{2}{\left(x-\mu \right)}^T{\Sigma }^{-1}\left(x-\mu \right)\right\}$$

Where:

• $\mu$ : vector mean
• $\sigma$ : variance matrix

Definition of Mean and Variance:

$$\begin{array}{rl}& E[x]={\int }_{-\infty }^{+\infty }xp(x)dx=\mu \\ & \mathrm{Var}[x]\triangleq E[{\left(x-\mu \right)}^2]={\int }_{-\infty }^{+\infty }{\left(x-\mu \right)}^{2}p(x)dx=\sigma \end{array}$$

Definition of Vector Mean and Variance Matrix (Multivariate):

$$\begin{array}{rl}& E[\underline{x}]=\underline{\mu }\\ & E[(\underline{x}-\underline{\mu })(\underline{x}-\underline{\mu }{)}^T]=\Sigma \end{array}$$

Distribution of Gaussian Data: If we plot the eigenvector of the variance matrix $\Sigma$ in a cartesian plane we obtain:

• The quantity $(\underline{x}-\underline{\mu }{)}^T{\Sigma }^{-1}(\underline{x}-\underline{\mu })$ is known as Mahalanobis distance between $\underline{x}$ and $\underline{\mu }$ , the hyper-ellipsoid we see in the graph are the level curves where $p(x)$ is constant, hence where the Mahalanobis distance is constant.

Special Case: The variance matrix $\Sigma$ is diagonal:

Special Case: The variance matrix $\Sigma ={\sigma }^{2}I$:

---

Original Files