TABLE OF CONTENTS A Markov process visits a state for a random sojourn time having an exponential distribution before jumping to the next state. These processes can be used to approximate Markov chains just as the Poisson process approximates the Bernoulli process. The advantage of the approximation is that we can often give the transient behavior of the Markov process explicitly. Markov processes are commonly used for describing queues when customers arrive according to Poisson processes or when service time distributions are exponential. Networks of such queues are widely used to model manufacturing and telecommunication systems and we will at least get an introduction to this interesting area of queueing networks.
We let X( t) represent the state at time t measured in seconds in a countable state space S which we may take to be {0, 1, 2, }. We proceed as with the Poisson process; that is, we approximate a discrete time Markov chain by a continuous time Markov process. The notion of norms described below will be very useful when we try to measure the accuracy of the approximation!
We shall assume that time is measured in multiples of a time unit which may be taken to be nanoseconds. We keep the notation developed in Chapter 4 and use square brackets to indicate both rounding up to the next integer and that a measurement is in nanoseconds. Any time t measured in seconds is denoted by 
when measured in nanoseconds.
Functions defined...
