TABLE OF CONTENTS Iterative testing throughout the development cycle is of fundamental importance when designing a multimodal interface. Empirical evaluation and user modeling are the proactive driving forces during system development.
In this section, the generic functionality required for successfully testing a multimodal interface is described. An initial step in testing an interface is the collection of user data for analysis. Given the need to examine a variety of modalities when designing a multimodal interface, an appropriate data collection infrastructure is required (Section 12.5.1). A strategy that has proven very fruitful for prototyping new multimodal interfaces is to exploit high-fidelity simulation techniques, which permit comparing trade-offs associated with alternative designs (Section 12.5.2). Analysis of multimodal data also requires synchronizing multiple data streams and development of annotation tools (as described in Section 12.5.3).
The focus of data collection is on the multimodal language and interaction patterns employed by users of a system. A data collection facility must therefore be capable of collecting high-quality recordings of multiple streams of synchronized data, such as speech, pen input, and video information conveying body motions, gestures, and facial expressions.
Besides producing recordings for further analysis, the collection infrastructure also has to provide facilities for observation during the collection in order to support simulation studies. Since views of each modality are necessary during some simulations, capabilities for real-time data streaming, integration, and display are required. Building a capable infrastructure to prototype new multimodal systems requires considerable effort (Arthur et...
