Neuronal dynamics of everyday activities and their Implications for robot control – an FMRI study

As part of the EASE CRC for the development of robots capable of autonomously performing everyday activities inspired by human abilities of spatial interaction and information processing, an fMRI study with 30 participants was conducted using a 3 Tesla Skyra MR-scanner. Stimulus material consisted of 1st-person videos of table setting activities that were annotated according to ontological event categories and that participants viewed imagining being the acting protagonist of the presented events, employing the brain’s inherent capabilities of motor imagery and action observation in order to closely simulate real word interaction on a neuronal level.
Resulting data were analyzed through General Linear Model (GLM) and Independent Component Analyses (ICA).
Through GLM-analyses, an event dependent allocation of neuronal activity was observed for event categories of object interaction and navigation, with a common baseline activity encompassing a network of executive functions in the frontal lobe combined with extensive activation in the occipital-, temporal- and parietal lobes within areas responsible for object recognition, interaction, and spatial cognition. Recruitment of supplementary areas was dependent on the experienced event category, with object interaction events resulting in increased activity within frontal areas as well as object identity- and action-related functions of the temporal and parietal lobes. Events of navigational context activated areas in the temporal-, parietal- and limbic- and occipital lobes responsible for topographical memory, spatial navigation, and whole-scene perception.
ICA found task-relevant components based on their temporal correlation with event categories, and the spatial characteristics of these components in relation to neuronal activity complemented results from the GLM-analyses. The waveform of selected components was furthermore used as basis for exploratory, semi-automated event recognition scripts.
Summarized, overall results could clearly demonstrate the viability of extracting event-related brain activity from complex, naturalistic stimuli.