In the previous section, we have seen that simple nonlinearities can be used very effectively in control systems. In this and the following sections, we will discuss some techniques based on nonlinearities, where the key idea is to use functions of several variables. It is not easy to characterize such functions in a simple way. The ideas described have been introduced under the names of neural and fuzzy control. At first sight these methods may seem quite complicated, but once we strip off the colorful language used, we will find that they are nothing but nonlinear functions.

Figure 7.20 Schematic diagram of a simple neuron.

Neural Networks

Neural networks originated in attempts to make simple models for neural activity in the brain and attempts to make devices that could recognize patterns and carry out simple learning tasks. A brief description that captures the essential idea follows.

A Simple Neuron

A schematic diagram of a simple neuron is shown in Figure 7.20. The system has many inputs and one output. If the output is y and the inputs are u₁, u₂, ... , u_n the input-output relation is described by

where the numbers w_i are called weights. The function f is a so-called sigmoid function, illustrated in Figure 7.21. Such a function can be represented as

where a is a parameter. This model of a neuron is thus simply a nonlinear function. Some special classes of functions can be approximated by Equation (7.10).

Neural Networks

More complicated models can be obtained by connecting neurons together as shown in Figure 7.22. This system is called a neural network or a neural net. The adjective feedforward is often added to indicate that the neurons are connected in a feedforward manner. There are also other types of neural networks. In the feedforward network, the input neurons are connected to a layer of neurons, the outputs of the neurons in the first layer are connected to the neurons in the second

Figure 7.21 Sigmoid functions.

Figure 7.22 A feedforward neural network.

layer, etc., until we have the outputs. The intermediate layers in the net are called hidden layers.

Each neuron is described by Equation (7.10). The input-output relation of a neural net is thus a nonlinear static function. Conversely we can consider a neural net as one way to construct a nonlinear function of several variables. The neural network representation implies that a nonlinear function of several variables is constructed from two components: a single nonlinear function, the sigmoid function (7.11), which is a scalar function of one variable; and linear operations. It is thus a simple way to construct a nonlinearity from simple operations. One reason why neural networks are interesting is that practically all continuous functions can be approximated by neural networks having one hidden layer. It has been found practical to use more hidden layers because then fewer weights can be used.

Learning

Notice that there are many parameters (weights) in a neural network. Assuming that there are n neurons in a layer, if all neurons are connected, n₂ parameters are then required to describe the connections between two layers. Another interesting property of a neural network is that there are so-called learning procedures. This is an algorithm that makes it possible to find parameters (weights) so that the function matches given input-output values. The parameters are typically obtained recursively by giving an input value to the function and the desired output value. The weights are then adjusted so that the data is matched. A new input-output pair is then given and the parameters are adjusted again. The procedure is repeated until a good fit has been obtained for a reasonable data set. This procedure is called training a network. A popular method for training a feedforward network is called back propagation. For this reason the feedforward net is sometimes called a back-propagation network.

Control Applications

A feedforward neural network is nothing but a nonlinear function of several variables with a training procedure. The function has many parameters (weights) that can be adjusted by the training procedure so that the function will match given data. Even if this is an extremely simplistic model of a real neuron, it is a very useful system component. In process control we can often make good use of nonlinear functions. Sensor calibration is one case. There are many situations where an instrument has many different sensors, the outputs of which must be combined nonlinearly to obtain the desired measured value. Nonlinear functions can also be used for pattern recognition.