This book represents a synergistic combination of stochastic processes, estimation, optimization and the analysis of recursive stochastic algorithms. Readers with varying interests and mathematical backgrounds may easily approach our book. This book deals with powerful and convenient approaches to a great variety of problems. It represents a contribution to the field of applied probabilities and statistics. We have written this book from an engineering point of view. Our emphasis here is on the design of techniques that enable the reader to use the theory.

The first chapter is devoted to stochastic processes. We present the foundations of probability theory: the conditional mathematical expectation, the notion of convergence for stochastic processes, different examples of stochastic processes related to different engineering areas, renewal processes, finite Markov chains and martingales, which have extensive applications in stochastic problems and arise naturally whenever one needs to consider mathematical expectations with respect to increasing information patterns. Many examples are given in order to illustrate, and to help readers deeply understand, the mathematical tools contained in this chapter. We have tried to make the contents easily accessible also for readers not familiar with mathematical works, and avoided using very complex mathematical tools. For example, many people are familiar with the classical concept of integrals in the Riemann sense but they know nothing about integration in the Lebesgue sense. We have avoided the use of measure theory. We have also introduced examples in order to explain physically the concepts that are useful in stochastic processes. Chapter 1 also...