Fast Recap:

Recap:

CNN (Competitive Neural Networks): ARCHITECTURE: In the output layer, there are 2 more type of connections:

Lateral connection (each output neuron is connected to each other output)
Self-connections (each output neuron has a connection that goes from itself to itself)

The lateral connections are inhibitory (their weights are $< 0$ ) While the self-connections are excitatory (their weights are $> 0$ )

Also at the end of the network there is a MAXNET component that realizes a winner takes all strategy, only one of the output units (a cluster) wins over the others (the losers are set to $0$ ).

DYNAMICS: Simple dynamics, the input is passed to the network, the network spits out the outputs ( $\overset{y}{^}_{i}$ for $i = 1, \dots$ ) then the MAXNET selects the winner and turn off all the other outputs.

LEARNING: On-line learning that relies on the following rule:

Δ w_{ij} = η (x_{j} - w_{ij})

NOTE: This formula is “theoretically grounded”. Ergo “It’s a good learning formula, we don’t need to search for another one”

Here we see an example of the learning of a CNN:

Where does the learning formula of the CNN comes from ?

We need to define $n_{i} (t)$ as a function dependent on time, since being a on-line learning algorithm $n_{i}$ will change possibly at each iteration.
In our case the vector $\underline{μ_{i}} (t)$ is the vector of the weights $\underline{w}$ of the network.

So $n_{i} (t)$ will become equal to: $n_{i} (t) + 1$
$\sum_{j = 1}^{n_{i} (t)} \underline{x_{j}}^{(i)}$ will become equal to: $\sum_{j = 1}^{n_{i} (t)} \underline{x_{j}}^{(i)} + \underline{x_{n_{i} (t) + 1}^{(i)}}$

We can simplify in the second passage $\frac{n _{i} ( t )}{n _{i} ( t ) + 1} \to 1$ since $n_{i} (t) ≫ 1$

Can we train a feed-forward neural network $ϕ (\underline{x})$ in an unsupervised manner, to estimate a PDF $p (\underline{x})$ ? ⇒ Yes, we can, but it’s tricky

We need to make the cost function:

C (τ, W) = p (τ ∣ W)

Where $p (\underline{x})$ is the output of our neural network. This formula means thatTODO

🪴 Quartz 4.0

Explorer

AI - Lecture 20

Fast Recap:

Recap:

Original Files:

Graph View

Backlinks