University AI - Supervised Learning - Non-Parametric Estimation

Supervised Learning: Non-Parametric Estimation

Up until now we have always made assumption on the form of the PDF when estimating the probability $P({\omega }_i|\underline{x})$.

To sum it up:

• Our objective is to estimate

$$P({\omega }_i|\underline{x})=\frac{p(\underline{x}|{\omega }_i)\cdot P({\omega }_i)}{p(\underline{x})}$$

• We don’t want to make any kind of assumption on the underling PDF of the data.

We can opt for one of three options:

1. Estimate the pdf $p(\underline{x}|{\omega }_i)$ using the data we have available, then calculate $P({\omega }_i|\underline{x})$ using the formula above.
2. Calculating directly $P({\omega }_i|\underline{x})$ using the data, which is the result of a chosen discriminant function.
3. We first reduce the dimension space of the data, $\phi :\mathbb{X}\to \mathbb{Y}$, then we estimate the probability using one of the other two methods ($1.$ or $2.$), but this time we make it using $\underline{y}=\phi (\underline{x})$ Computing $P({\omega }_i|\underline{y})$ turn out to be easier than computing $P({\omega }_i|\underline{x})$

---

Original Files

The idea of projecting $\mathbb{X}$ on $\mathbb{Y}$ is the same as in the Unscented Kalman Filter and the Particle Filter, creating a pdf given some data: