More than a buzz: advancing haptic feedback in virtual reality and mixed reality

Touching an object elicits rich haptic sensations that reveal its physical properties to the haptic sense.
For instance, when grasping a coffee cup, humans can feel its warmth and sense the coffee shifting
inside. Replicating such haptic sensations is fundamental to immersive virtual reality (VR) and mixed
reality (MR), as it allows users to feel virtual objects during touch interactions. While this claim for
haptic richness is reflected in the theoretical definition of immersion, in practice, creating such vivid
haptic sensations in digital interactions is challenging. Advances in displays and computing have
made consumer VR and MR more portable and lightweight, but have also constrained their haptic
capabilities. In practice, these systems are limited to vibrotactile feedback via handheld controllers or
provide none at all in hand-tracking scenarios. Although prior research has proposed a range of haptic
technologies to deliver richer haptic experiences, most approaches rely on specialised hardware,
which creates barriers to widespread adoption. As a result, widely available consumer VR and MR
systems fall short of the sensory richness that immersion strives to provide, as they offer either no
haptic feedback or only a narrow range of tactile sensations, omitting kinesthetic information.

This dissertation addresses this gap by exploring novel ways to support vivid haptic experiences of
virtual objects while reducing reliance on specialised hardware. It advocates for leveraging established
affordances of current and future off-the-shelf VR and MR systems, mobile phones, and everyday
objects. To this end, it investigates several approaches. First, it shows how the haptic capacities of VR
controllers can be enhanced through software-based pseudo-haptic illusions. Second, it demonstrates
the potential to substitute specialised haptic technology with passive proxies augmented by headbased
haptic actuation to convey surface sensations. Third, it examines the potential of mobile phones
as a makeshift haptic interface for MR systems that often lack tangible interaction devices. This
work offers a design space for using physical features for passive haptic feedback and an empirical
understanding of how users hold and contact mobile phones during use. Findings offer practical
implications for redesigns of future mobile phones.

This cumulative dissertation is structured in four parts. Part I provides a synopsis, motivating the
topic, presenting the research questions and methodologies, and outlining the contributions. Part II
provides the theoretical foundation of haptic perception and reviews related work. Part III contains
the five included publications. Part IV discusses the implications, limitations, and directions for future
research for advancing haptic feedback in VR and MR.

Given the persistent gap between the specialised haptic technology proposed in prior research
and the continued reliance of consumer systems on vibrotactile feedback, this dissertation uncovers
new design spaces for advancing haptic feedback in VR and MR to enable richer haptic experiences.
Its multi-angle approach contributes empirical findings, artefacts, methodologies, and datasets that
can inform the development and evaluation of future haptic interfaces. This dissertation aims to
encourage further research bridging this gap by reimagining established affordances of current and
future technologies to enrich immersive haptic experience.