An engagement-aware recommender system for people with dementia

Abstract

With the number of people with dementia (PWD) growing worldwide and no cure available, there is an urgent need for tertiary prevention strategies to reduce symptoms, slow disease progression, and improve patients' quality of life. Technology-driven approaches, such as tablet-based activation systems, are well-suited for this purpose as they are portable and can offer diverse contents to meet different capabilities, needs, and interests. However, for these systems to benefit cognitive functioning, well-being, and quality of life, they must engage users effectively, which varies greatly among individuals. Thus, tablet-based systems should be able to monitor and interpret users’ engagement levels to personalize the sessions. This thesis examines a tablet-based system called I-CARE which aims to promote cognitive, social, and physical activation for PWD. It analyzes interactions between PWD and their caregivers (CGs) using a multimodal dataset gathered in a semi-supervised care setting, with engagement levels annotated by human raters for detailed analysis. The dissertation explores the recognition of PWD's affective states, including engagement and activation preferences, based on their verbal and non-verbal responses. Moreover, recognized engagement levels are used as feedback for a recommender system designed to support PWD and CGs in selecting appropriate contents, enabling beneficial activation sessions. A study conducted in a real-world setting evaluated the system's effectiveness and identified challenges in dynamic scenarios. Last, this thesis investigates the automatic detection of spontaneous emotional tears, a key social signal, based on facial and postural behaviors in healthy individuals, which could enhance perceptive abilities across various applications.