Empowering surgeons - a multimodal interaction approach to improve computer-assisted surgical planning and intervention

Abstract

Advancements like X-ray, magnetic resonance imaging (MRI), computer tomography (CT), and infrared needle tracking improved diagnostics and raised survival rates in the medical field. These computer support systems are an integral part of contemporary treatment. Technological advancements such as 3D visualisations, virtual reality and similar technologies improved work-life in many areas, e.g., architecture, the film industry or specialised training for pilots. In contrast, medical staff still uses "rudimentary" devices such as a desktop PC with a mouse and keyboard to interact with the 2D images from scans. Technologies with high usability and user experience are sparse.

The prototypes presented in this thesis focus on empowering surgeons by improving the interaction with medical data and supporting procedures in the preoperative and intraoperative setting. In the scope of the research presented in this thesis, I introduce surgeons to tangibles, 3D models, virtual reality, gesture control, electroluminescence displays and new kinds of navigational feedback. In combination with methods from human-computer interaction, such as user-centred and participatory design, focus groups as well as user studies, I integrate these technologies into applications to empower surgeons rather than restrict or replace them.

Overall, the research presented in this thesis advocates using a human-computer interaction point of view and methods to resolve current technology-based issues in the medical domain. The research results show that these methods and approaches achieved better usability and user experience without increasing the perceived workload. The presented prototypes also increase the surgeons' autonomy and thereby provide empowerment to the surgeons.