What Is Optimization ?

All optimization problems are made up of three basic ingredients:

1. An objective function which we want to minimize or maximize. For instance, in a manufacturing process, we might want to maximize the profit or minimize the cost. In fitting experimental data to a user-defined model, we might minimize the total deviation of observed data from predictions based on the model. In designing an automobile panel, we might want to maximize the strength. The objective function should be quantitative.

2. A set of unknowns or variables which affect the value of the objective function. In the manufacturing problem, the variables might include the amounts of different resources used or the time spent on each activity. In fitting-the-data problem, the unknowns are the parameters that define the model. In the panel design problem, the variables used define the shape and dimensions of the panel.

3. A set of constraints that allow the unknowns to take on certain values but exclude others. For the manufacturing problem, it does not make sense to spend a negative amount of time on any activity, so we constrain all the "time" variables to be nonnegative. In the panel design problem, we would probably want to limit the weight of the product and to constrain its shape.

The optimization problem consists of finding values of the variables that minimize or maximize the objective function while satisfying the constraints and determining if all these ingredients are necessary.