TABLE OF CONTENTS We address here the problem of coding on a noisy channel. The goal is to encode information in order to protect it against transmission errors. We consider a source that is optimally or near optimally encoded as we have explained above. We add additional control symbols to the symbols generated on the channel, allowing the receiver to detect, or even to correct possible transmission errors. We then speak of error-detecting and error-correcting codes. Shannon's second theorem, or the noisy-channel coding theorem (1948), guarantees the existence of such codes.
The theorem, also called the fundamental theorem, is formulated as follows:
Consider a source with information rate R bits/s, and a channel with capacity C bits/s. Under the condition R < C, there exists a code with words of length n such that the maximum error probability at the decoder P E is P E ? 2 ?nE( R).
In the expression, E( R) is a non-negative function decreasing with R, called the random-coding exponent of the channel.
We do not give any proof of the theorem here, as it would not be of any use for engineers. We nevertheless illustrate its meaning, then in the following sections we will develop its applications.
In accordance with definition (5.25) the code rate R (the information rate) is equal to the entropy per symbol at channel input, when these symbols are...
