User modeling for adaptation of cognitive systems

Computer systems have been continuously evolving since the first computer
invented in the 1950’s up to the revolution of smart devices, which we are
witnessing nowadays. Consequently, the number of computer users has been
exponentially increasing, from only few experts to hundreds of millions users:
novices, intermediates, and experts.
Human Computer Interaction (HCI) has been widely discussed for improving
the interaction between a computer system and its users. Interdisciplinary
research evolved for integrating psychological studies with HCI studies to
model cognitive skills during HCI sessions, and thus, to design an effective
User Interface (UI). Due to the high dynamic nature of HCI sessions, a
dynamic UI adaptation is required when user performance is impaired.
User performance impairment in an HCI session should be detected "online",
i.e. during the system use, for an appropriate UI adaptation; It is valuable to
detect the reason which caused the performance impairment during an HCI
session, so-called HCI obstacle, because different UI adaptations are required
to compensate for different HCI obstacles.
Different HCI obstacles impair several human processes, namely perception
and cognition processes. Consequently, several human processes can be impaired during an HCI session. Human processes can be tracked by recording
appropriate multimodal data during an HCI session: brain activity data to
track the cognition process and behavioral data, depicted by encoded user
actions, to track the user behaviour.
Modeling of multimodal HCI obstacles is very important because there is no
"silver bullet" UI adaptation which could be activated by default. In other
words, while different HCI obstacles, e.g. memory-based and visual obstacles,
impair the user HCI performance, different UI adaptation mechanisms will suit
each individual HCI obstacle, because an UI adaptation should appropriately
compensate for the impaired human process. Moreover, a good UI adaptation
mechanism for a specific HCI obstacle can be detrimental if applied for other
HCI obstacles.
In this thesis, a novel user modeling based cognitive adaptive system is proposed. The adaptive system dynamically models memory-based and visual
HCI obstacles during system use, and accordingly applies the suitable UI
adaptation mechanism for each detected HCI obstacle. Appropriate machine
learning models are used for multimodal HCI obstacles detection. The multimodal obstacle detectors outputs are passed to an overarching probabilistic
model to decide for the most suitable UI adaptation mechanism.
The proposed approach is dynamic in consecutive HCI sessions, i.e. it treats
not only persistent HCI obstacles which remain impairing the user performance
in consecutive HCI sessions, but also volatile HCI obstacles which suddenly
appear or disappear in the HCI sessions. Moreover, the model is dynamic in
case of wrongly decided UI adaptation for an HCI session, where it recovers
in the subsequent sessions.
The approach is systemically evaluated through data collected from many user
studies, and the experimental results show that our approach: 1) models the
HCI obstacles well, where it can simulate the user behaviour under different
conditions, 2) detects multimodal HCI obstacles in consecutive sessions, and
3) dynamically learns from consecutive HCI sessions to accurately adapt the
UI.