TABLE OF CONTENTS Processes design is mainly based on modeling and simulation approaches. System simulation is the mimicking of the operation of real systems by a computer [1, 31]. We strive to build models not just for the fun of it, but to use the models for analysis, outcomes of which affect our decisions (strategies) in the future.
The modeling problem can be tackled in many ways, but here we shall be concerned with the use of finite Markov chains in order to achieve this objective. Uncontrolled and controlled finite Markov chains have been under study for a long time, and from several points of view, in physics, mathematics, engineering, economics, and many other fields [8, 14, 19, 32, 37, 43, 51]. But the subject is such a fundamental and deep one that there is no doubt that Markov chains will continue to be an object of study for as long as one can foresee. In [40], based on Markov chains, a basic model for performance evaluation of a simplified radio network controller system has been developed. Notice that, under certain conditions, many stochastic algorithms such as iterated random functions, stochastic approximation techniques, simulated annealing, genetic algorithms, Kalman-Bucy filter, etc., can be mathematically modeled using Markov chains [6].
This section is devoted to uncontrolled finite Markov chains. We are going to consider the main results, relevant in many areas. The next subsection introduces mathematical concepts that are fundamental to the remainder of the text. We begin with a general...
