Applications

In orther to have an integrated understanding on neural networks, we adopt the next perspective, called top-down, from application, algorithm to architecture:

The approach is application-motivated, theoretically-based, and implementation oriented. The main applications are for signal processing and pattern recognition. The algorithmic treatment represents a combination of mathematical theory and heuristic justification for neural models. The ultimate objective is the implementation of digital neurocomputers, embracing technologies of VLSI, adaptive, digital and parallel processing.

Applications

From an application driven perspective, one can see that the strengh of neural networks are nonlinear, adaptive and parallel processing. Neural networks have found many successful applications in computer vision, signal/image processing, speech/character recognition, expert systems, medical image analysis, remote sensing, robotic processing, industrial inspection, and scientific exploration. The application domains of neural nets can be roughly divided into the following categories: association, clustering, classifications, pattern completion, regression and generalization, and optimization.

Association, Clustering and Classification

In this application, input static patterns or temporal signals are to be classified or recognized. Ideally, a classifier should be trained such that when a slightly distorted version of stimulus is presented it can still be correctly recognized. Equivalently, the network should have a certain noise inmunity feature, that is, it should be able to recover a "clean" signal from noisy environments or channels. This is critical to holographic, associative, retrieval applications.

Association: Of special interest are the two association formulations: autoassociation and hetero-association. The autoassociation problem is to retrieve the complete pattern, given partial information of the desired pattern. The heterassociation is to retrieve a corresponding pattern set B, given a pattern in a set A. The weights in associative networks are often predetermined based on a Hebbian type formulation. Usually, the autocorrelation of the set of the stored patterns determined the weights in autoassociation networks. On the other hand, the crosscorrelation of multiple pairs of the patterns is used to determined the weights of a heteroassociation network.
Unsupervised Clustering: For this application paradigm, the synaptic weights of the network are trained by an unsupervised learning rule, that is, the network adapts the weights and verifies the result based exclusively on the input patterns.
Supervised Classification: The supervised classification adopts some forms of approximation or interpolatin criteria. In many classification applications, for example, speech recognition, the training data consist of pairs of input and output patterns. In this case, it more advantageous to adopt supervised networks such as the well known backpropagation network. It is more suitable for the applications problems, which involve a large number of classes with more complex separating borders.

Pattern Completion

In many classification problems, an implied task is information completion, that is, recover the original pattern given only partial information. There are two kinds of pattern completions problems: temporal and static. The proper use of contextual information is key to a successful recognition.

Regression and Generalization

Linear or nonlinear regression provides a smooth and robust curve fiting to training patterns. It can be extended to an interpolation problem. The system is trained by a large set of training samples based on a supervised learning procedure. A network is considered successfully trained if it can closely approximate the teacher values for the trained data space and can provide smooth interpolations for the untrained data space. The objective of generalization is to yield a correct output response to an input stimulus to which it has not been trained before. The system must induce the salient feature of the input stimuli and detect the regularity. Such regularity discovery ability is critical to many applications. It enables the system to function competently throughout the entire space, even though it has benn trained only by a limited body of exemplary patterns. See this figure:

Optimization

Neural nets offer an appealing tool for optimization applications, which usually involve finding a global minimum of an energy function:

Once the energy function is defined, then the determination of the synaptic weights is relatively straightforward. In some applications, the energy function is directly available. In some others, the energy function must be derived from the given cost criterion and special constraints. A major difficulty associated with the optimization problem is the high possibility of a solution converging to a local optimum instead of the global optimum. To battle the problem, several, several stattistical techniques are proposed, for example, stochastic simulated annealing (SSA) and mean field annealing.

UNSUPERVISED MODELS

Contents

Artificial Neural Networks

About this Tutorial