Embedded Media Processing

As consumers, we're intimately familiar with video systems in many embodiments. However, from the embedded developer's viewpoint, video represents a tangled web of different resolutions, formats, standards, sources and displays. Many references exist that delve into the details of all things video; outstanding among them are References 33 and 34 in the Appendix.
In this chapter, our goal will not be to duplicate such excellent works. Rather, we will strive to untangle some of this intricate web, focusing on the most common circumstances you're likely to face in today's media processing systems. After reviewing the basics of video, we will discuss some common scenarios you may encounter in embedded multimedia design and provide some tips and tricks for dealing with challenging video design issues.
Let's start by discussing a little physiology. Understanding how our eyes work has paved an important path in the evolution of video and imaging. As we'll see, video formats and compression algorithms both rely on the eye's responses to different types of stimuli.
Our eyes contain two types of vision cells: rods and cones. Rods are primarily sensitive to light intensity as opposed to color, and they give us night vision capability. Cones, on the other hand, are not tuned to intensity, but instead are sensitive to wavelengths of light between 400 nm (violet) and 770 nm (red). Thus, the cones provide the foundation for our color perception.
There are three types of cones, each with a different pigment that's either most...