Supervised
and Unsupervised Neural Networks
The neural networks are commonly categorized in terms of their corresponding
training algorithms: fixed-weights
networks, unsupervised networks,
and supervised networks. There is
no learning required for the fixed-weight networks, so a learning mode
is supervised or unsupervised.
Supervised Learning Rules
Supervised learning networks have been the mainstream of neural model development.
The training data consist of many pairs of input/output training pattens.
Therefore, the learning will benefit from the assistance of the teacher.
Given a new training pattern, say, (m+1)th, the weights may be updated
as follows:
-
You can see an schematic diagram of a supervised system in next figure:
-
Unsupervised Learning Rules
For an unsupervised learning rule, the training set consist of input training
patterns only. Therefore, the network is trained without benefit of any
teacher. The network learns to adapt based on the experiences collected
through the previous training patterns. Here is a tipical schema of an
unsupervised system:
-
Typical examples are the Hebbian learning rule, and the competitive
learning rule.
A simple version of Hebbian learning rule
is that when unit i and unit jare simultaneously excited,
the strength of the connection between them increases in proportion to
the product of their activations.
As an example of competitive learning,
if a new pattern is determined to belong to a previously recognized cluster,
then the inclusion of the new pattern into that cluster will affect the
representation (e.g., the centroid) of the cluster. This will in turn change
the weights characterizing the classification network. If the new pattern
of ios determined to belong to none of the previously recognized cluster,
then (the structure and the weights of) the neural network will be adjusted
to accommodate the new class (cluster).
Basis Function and Activation Function
Contents
Artificial Neural Networks
About this Tutorial