TABLE OF CONTENTS Multimodal interfaces depend ultimately on the quality of the underlying recognizers for multiple modalities that are used. Thus, these interfaces benefit from the advances of natural language processing techniques and advances in hardware capabilities that make it possible for more challenging recognitions to be successfully achieved.
Mobile computing is one of the areas in which multimodal interfaces are expected to play an important role in the future. Multimodal interfaces suit mobility particularly well because of the small factor requirements that are usually imposed by on-the-move operation. Multimodal interfaces provide expressive means of interaction to users without requiring bulky, hard-to-carry equipment such as keyboards or mice.
The flexibility provided by interfaces that support multiple modes also fits the demands introduced by mobility via the adaptation to shifting contexts of use. A multimodal interface can, for instance, provide spoken operation for users whose hands and eyes are busy, such as while driving, or maintaining situational awareness in dangerous environments, such as disaster areas. These interfaces can also adapt to noisy environments either by leveraging mutual disambiguation to enhance recognition or by providing means for commands to be given via nonspoken means such as a pen.
Speech and pen input are already supported by a variety of devices (smart phones, PDAs), and successful commercial implementations are available. Further computational power should lead to an era in which a majority of the mobile devices might offer multimodal capabilities, including increasing levels of video-based modalities.
Collaborative...
