Digital Watermarking
Learn about this key ingredient to copyright protection.
TABLE OF CONTENTS Digital Watermarking
Appendix A: Background Concepts
A.1 Information Theory
In our discussion of message coding with side information, we use several concepts from information theory. For those readers unfamiliar with information theory, we now present a brief description of each of these concepts. Our intention in this appendix is to develop some intuition for these concepts, rather than provide a rigorous analysis. Those interested in more detailed discussions of these ideas are referred to one of the many books on information theory, such as [54, 97].
A.1.1 Entropy
The entropy of a random variable, x, is a value that gives some idea of how random that variable is. It is defined, mathematically, as
| (A.1) |  |
where P x( x) is the probability that x will be equal to x. To compute entropy when P x( x) = 0 for some value of x, we specify that 0 log 2 0 = 0. If x is a continuous variable (i.e., real-valued), the summation in Equation A.1 can be replaced with integration, and the probability distribution with a probability density function.
Intuitively, the entropy of x is the average number of bits of information required to identify its value. An example illustrates this idea. Imagine that two people, Alice and Bob, play a simple game with a deck of four cards. The backs of the cards are all identical to one another (as in a normal deck). The face of each card is labeled with a...
Copyright Academic Press 2002 under license agreement with Books24x7
