TABLE OF CONTENTS We now address the issue of transmitting the information through a perturbed medium. Just as for the source, we will rely on the concept of entropy, and more specifically here on conditional and mutual entropy, from which we can define the information capacity of a noisy channel. Then the following sections introduce the second Shannon's theorem and the methods for detecting and correcting errors, so as to increase the information capacity of the channel.
Generally, we mean by channel any physical or logical entity used for carrying and processing information (transmission channel in a network, bus in a computer, data processing system, etc.). Formally, noisy channels operate a transformation between the set of input symbols and the set of output symbols.
We concentrate on discrete (binary), memoryless (the transformation of one input symbol into another does not depend on previous transformations) and stationary channels (the transformation does not depend on the time origin).
[ X] denotes the set of all possible input symbols, and [ Y] the set of possible symbols at the output. We associate each input symbol x i with its occurrence probability p( x i), and similarly p( y j) is associated with output symbol y j. Due to the perturbations, sets [ X] and [ Y] may differ, and so do p( y j) and p( x i).
The channel...
