TABLE OF CONTENTS The very title of this book is borrowed from the information theory vocabulary, and, quite naturally, it is an outline of this theory that will serve as an introduction. The subject of information theory is the scientific study of communications. To this end it defines a quantitative measurement of the communicated content, i.e. information, and deals with two operations essential for communication techniques: source coding and channel encoding. Its main results are two fundamental theorems related to each of these operations. The possibility of channel encoding itself has been essentially revealed by information theory. That shows, to which point a brief summary of this theory is essential for its introduction. Apart from some capital knowledge of its possibilities and limits, the theory has, however, hardly contributed to the invention of means of implementation: whereas it is the necessary basis for the understanding of channel encoding, it by no means suffices for its description. The reader interested in information theory, but requiring more information than is provided in this brief introduction, may refer to [1], which also contains broader bibliographical references.
To start with, we will comment on the model of a communication, known as the Shannon paradigm after the American engineer and mathematician Claude E. Shannon, born in 1916, who set down the foundations for information theory and established the principal results [2], in particular, two fundamental theorems. This model is represented in Figure 1.1. A source generates a message...
