TABLE OF CONTENTS This chapter was devoted to stochastic processes. We presented and described probability space. The expectation as well as the conditional mathematical expectation were given and their properties were reported. Many examples such as random walk, Markov processes, Markov chains, renewal processes and martingales were presented. We frequently considered the sum of random variables, which plays an important role in many engineering areas. We recalled the main operations on states and showed their connection with probability calculus. Martingales arise naturally whenever one needs to consider mathematical expectations with respect to increasing information patterns.
The detailed proofs of the theoretical results presented in this chapter can serve as the first step in a training process for the analysis of recursive schemes. In this sense, this chapter represented a kind of initial step to Chapter 4.
H. Arsham, Systems Simulation: The Shortest Path from Learning to Applications, http://home.ubalt.edu/ntsbarsh/index.html/
B. B. Ash, Real Analysis and Probability, Academic Press, New York, 1972.
K. J. str m, Introduction to Stochastic Control Theory, Academic Press, New York, 1970.
R. E. Barlow, Engineering Reliability, SIAM Classics in Applied Mathematics, Philadelphia, 1998.
R. E. Barlow and F. Proschan, Mathematical Theory of Reliability, SIAM Classics in Applied Mathematics, Philadelphia, 1996.
N. Bartoli and P. Del Moral, Simulation algorithmes stochastiques, C padu s-Editions, Toulouse, 2001.
S. Beveridge and C. Nelson, A new approach to decomposition of economic time series into permanent and transitory components with particular attention to measurement of the business cycle, Journal of...
