Parallel (Synchronous) Hopfield Model

The sequential Hopfield model suffers from a disadvantage in its disallowing parallel updating. In orther to exploit the potential of parallel (i.e., synchronous) processing, the following model is proposed.

Derivation of Weights in the parallel Hopfield Model In the parallel model, the weights are defined as

Note that the diagonal weights  are no longer set to zero. The thresholds of the network are given as
Algorithm

 During the kth iteration:

  1. Compute the net values in parallel for i = 1,2, ..., N,
  2. Update the states in parallel for i=1,2, ...,N,
    3. Repeat the same process for the next iteration until convergence, which occurs when none of the elements changes state during any iteration.
     
     
Convergence : Gradual Decrease of Energy The proof of convergence for this parallel Hopfield model is presented here. For the synchronous update, assume that the kth parallel iteration, we have an energy function E(k):
According to
the energy level change due to one iteration of parallel update is
Because the W matrix is formed by the outer product without diagonal nullification, it is a nonnegative definite matrix, that is, . It should be clear now that . More precisely,  if a nontrivial update occurs. (This also assures that there is no possibility of oscilation of any convergent state.)

Competitive Learning Networks

Contents

Artificial Neural Networks
About this Tutorial