Overview

Rama Chellappa and Shaohua Kevin Zhou

Center for Automation Research
University of Maryland
College Park, MD 20742, USA
rama@cfar.umd.edu

Siemens Corporate Research
755 College Road East
Princeton, NJ 08540, USA
kzhou@scr.Siemens.com

Most face recognition algorithms take still images as probe inputs. This chapter presents a video-based face recognition approach that takes video sequences as inputs. Because the detected face might be moving in the video sequence, we inevitably have to deal with uncertainty in tracking as well as in recognition. Rather than resolving these two uncertainties separately, our strategy is to perform simultaneous tracking and recognition of human faces from a video sequence.

In general, a video sequence is a collection of still images; so still-image-based recognition algorithms can always be applied. An important property of a video sequence is its temporal continuity. Although this property has been exploited for tracking, it has not been used for recognition. In this chapter, we systematically investigate how temporal continuity can be incorporated for video-based recognition.

Our probabilistic approach solves still-to-video recognition, where the gallery consists of still images and the probes are video sequences. A time series state space model is proposed to fuse temporal information in a probe video, which simultaneously characterizes the kinematics and identity using a motion vector and an identity variable, respectively. The joint posterior distribution of the motion vector and the identity variable is estimated at each time instant and then propagated to the next time instant. Marginalization over the motion vector yields a robust estimate...