Device-dependant modality selection for user-interfaces - an empirical study

The presentation of information using multiple modalities influences the perception of users, their comfort, and their performance in using a computer-based information system. This paper presents a user study investigating the effects of different output modality-combinations on the effectiveness to transport information and on the user's acceptance of the system. We chose a tourist information system as a test environment and conducted the study on three different devices (PDA, TV set, and desktop computer) to investigate whether the best modality-combination depends on the used device. It turned out that the modality-combination of spoken text in connection with a picture was the most effective regarding recall-performance. This effect was strongest for users working with PDAs, which can be explained by the cognitive load theory. In contrast to this, participants ranked different modality combinations as most appealing, namely those with written text.