Control Theory, Second Edition

Optimisation is concerned with finding the best possible solution, formally referred to as the optimal solution, to a particular problem. The term optimisation is often used very loosely in general speech but in control theory it has a precise meaning: the action of finding the best possible solution as defined by an unambiguous criterion (or cost function).
Optimisation has, to some extent deservedly, acquired a reputation for being out of touch with reality. This is because the analytic techniques for optimisation are highly involved and in order to make headway many workers have resorted to drastic modification of the original problem to allow application of some particular optimisation technique; i.e. simplistic assumptions about the problem have, unsurprisingly, produced simplistic solutions. Currently, more healthy attitudes are beginning to prevail. For instance, it is becoming accepted that, for large complex problems, it may be better to encode optimality criteria in more vague but more realistic terms than parallel human evaluation criteria, than to force unwilling problems into an ill-fitting straitjacket to allow rigorous optimisation. With these reservations having been made, it is possible to turn to the ideas and techniques of optimisation theory and practice.
If we know the 'formula' for the function f, the maximum value can be found by the methods of elementary calculus.
If f is not known as a...