TABLE OF CONTENTS In recent decades the efforts of an increasing number of researchers have led to the development of a wide variety of numerical methods for optimization purposes. Optimization techniques are attracting increasing interest from researchers with various backgrounds (statistics, chemical engineering, automatic control, mechanics, maintenance, etc.) and play an increasingly important role in many areas (engineering, finance, marketing, insurance, etc.). Many industry analysers believe that the emphasis in the near future will be on improving efficiency and increasing profitability of existing plants, rather than on plant expansion.
Optimization techniques stem from diverse approaches, frequently grounded on heuristic intuitions, physical or biological considerations. An extensive literature on optimization techniques exists [6, 7, 14, 24, 33, 47, 50, 53, 58, 59, 61].
In this chapter we shall be concerned with optimization algorithms used in the framework of many engineering and economic problems, and with aspects related to their implementation. The development of optimization strategies in different areas using microcomputers requires algorithms that are both numerically economical and robust. We shall present four techniques for the minimization of both unconstrained and constrained problems (cost functions and constraints), multimodal functions (global optimum) and mixed integer programming problems. The algorithms presented in this chapter, namely, stochastic approximation techniques (SAT), learning automata (LA), simulated annealing (SA) and genetic algorithms (GA), all belong to the class of random search techniques. Random search techniques lead readily to global search with the advantages of simplicity, efficiency and flexibility.
Let us start by simple consideration of extrema. A function
