Taxonomy of Neural
Networks
There are two phases in neural information processing. They are the learning
phase and the retrieving phase. In the training phase, a training
data set is used to determine the weight parameters that define the neural
model. This trained neural model will be used later in the retrieving phase
to process real test patterns and yield classification results.
-
Retrieving Phase: Various nonlinear systems
have been proposed for retrieving desired or stored patterns. The results
can be either computed in one shot or updated iteratively based on the
retrieving dynamics equations. The final neuron values represent the desired
output to be retrieved.
-
Learning Phase: A salient feature of neural
networks is their learning ability. They learn by adaptively updating the
synaptic weights that charcterize the strength of the connections. The
weights are updated according to the information extracted from new training
patterns. Usually, the optimal weights are obtained by optimizing (minimizing
or maximizing) certain "energy" functions. For example, a popular criterion
in supervised learning is to minimized
the least-squares-error between
the teacher value and the actual output value.
Real-world applications may face two very different kinds of real-time
processing requirements. One requires real-time retrieving but off-line
training speed. The other demands both retrieving and training in real-time.
These two lead to very different processing speeds, which in turn affect
the algorithm and hardware adopted.
A possible taxonomy of neural networks is;
Neural Networks
Fixed Unsupervised Supervised
Hamming Net Neocognitron Perceptron
Hopfield Net Featuer Map Decision-Based NN
Combinatorial Competitive Learning ADALINE (LMS) Optimization
ART Multilayer Perceptron
Principal Component Temporal Dynamic Models
Hidden Markov Model
Supervised and Unsupervised Networks
Contents
Artificial Neural Networks
About this Tutorial