Unraveling the fidelity of virtual reality interactions: effects of realism in object manipulation and embodiment

Abstract

Immersive experiences created with multisensory and interactive virtual reality (VR) technology can feel incredibly realistic. Many factors of the system and the user determine the fidelity of VR interactions, that is, how closely an interaction resembles the reference interaction to be simulated. The human-computer interaction (HCI) research presented in this dissertation is a step towards unraveling and better understanding the different aspects of interaction fidelity in VR that cover user input, computer simulation, sensory feedback, and the user's experience. Combining theoretical, methodological, and empirical research, I present and discuss nine papers focusing on object manipulation and embodiment in the context of interaction fidelity. The presented Interaction Fidelity Model enables a systematic assessment of eight fidelity components with precise definitions and a thorough discussion of their implications. Integrating the HCI loop and previous theories, the model's human-centered perspective can be universally applied to all interactions with any VR system. Furthermore, the specialized Haptic Fidelity Framework allows a detailed assessment of 14 factors determining the fidelity of haptic interfaces. On this theoretical basis, a series of user studies are analyzed, covering findings on haptic feedback and perception for object manipulation, body visibility in VR sports, visual realism of virtual humans, embedding questionnaires into virtual environments for user research, and VR meetings of an academic team. The quantitative and qualitative analyses demonstrate significant effects of varying interaction fidelity on user performance, task load, sense of body ownership, perceived realism, usability, and user experience. The iterative user research contributes hardware and software prototypes, such as an in-VR questionnaire tool for participants' self-reports, a grip-based interaction technique for force-feedback gloves, a handheld controller with adaptive trigger resistance, and experimental environments built with Unity.

This thesis provides guidelines, illustrative examples, research artifacts, and educational material for designing realistic VR applications, improving how we comprehend and reflect on interaction fidelity in VR research and development. An extensive research agenda highlights promising directions for future investigations of the complex construct of interaction fidelity.