What we really want from a fast computer is to quickly run the programs that we care about. Better performance simply means less program run time.

To improve performance, we must increase frequency or average instructions per cycle (IPC) or reduce the number of instructions required. Choices in architecture will affect the IPC and instruction count. Choices in microarchitecture seek to improve frequency or IPC. Performance is improved by increasing either, but in general, changes that improve one make the other worse. The easiest way to improve frequency is by increasing the pipeline depth, dividing the execution of each instruction into smaller faster cycles.

The examples earlier in this chapter had a pipeline depth of 4, dividing instructions into 4 cycles. If instructions are balls rolling down a pipe, this means the time between adding new balls to the pipe is equal to 1/4 the time a ball takes to roll the length of the pipe. Adding new balls at this rate means there will always be 4 balls in the pipe at one time. If a pipeline depth of 8 had been chosen instead, the time between adding new balls would be 1/8 the time to roll the entire length and there would be 8 balls in the pipe at one time. Doubling the pipeline depth has doubled rate balls are added by cutting in half the time between new balls.

The time between adding new balls to the pipe is equivalent to a processor's cycle...