Novel algorithms and methods for immersive telepresence and collaborative VR - RGB-D streaming, 3D scene (re-)construction, and avatar visualization

Abstract

This thesis is concerned with collaborative VR and tackles many of its challenges. A core contribution is the design, development, and evaluation of a low-latency, real-time point cloud streaming and rendering pipeline for VR-based telepresence that enables to have high-quality live-captured 3D scenes and avatars in shared virtual environments. Additionally, we combined a custom direct volume renderer with an Unreal Engine 4-based collaborative VR application for immersive medical CT data visualization/inspection. We also propose novel methods for RGB-D/depth image enhancement and compression and investigate the observer’s perception of new 3D visualizations for the potentially confusing teleport locomotion metaphor in multi-user environments. Moreover, we designed and developed multiple methods to efficiently procedurally generate realistically-looking terrains for VR applications, that are capable of creating plausible biome distributions as well as natural water bodies. Lastly, we conducted a study and investigated the effects of collaborative anatomy learning in VR. Eventually, with the ensemble of contributions that we presented within this thesis, including not only novel algorithms and methods but also studies and comprehensive evaluations, we were able to improve collaborative VR on many fronts and provide critical insights into various research topics.