Delta Learning Rule: ADALINE

The simplest approximation formulation is one based on the linear least-squares-error criterion. Indeed, one of the pioneering neural models is a linear network, named ADALINE (ADAptive LINear Element), proposed by Widrow. As tou can see in this figure, ADALINE is a linear single layer network with a net value

where

is the number of inputs nodes. Without los of generality, it is more convenient to regard the threshold

just as an extra weight, that is,

. The net value can be rewritten as

where

and

. Note that z is the augmented pattern x.

Least-Squares-Error Criterion

The weight vector w can be obtained by minimizing the least-squares-erros criterion:

The delta learning rule adopted in ADALINE is a data-adaptive technique for deriving a least-squares-error solution. It is based on an iterative gradient-descent algorithm. The gradient is defined as the first partial derivative of

with respect to

In a vector form:

In order to minimize E, the weights should be updated in the direction opposite to that of the gradient, that is,

Nonlinear Multilayer Backpropagation Networks

Contents

Artificial Neural Networks

About this Tutorial