Causal Observational Learning in Natural Settings for Robot Analysis of Human Interaction

Learning from observing others is a powerful human competence that robots lack. People can learn by analyzing others’ interactions under almost any conditions because of reasoning capabilities, such as inferring causal relationships, predicting, adapting, and imagining. These capabilities allow people to attain causal understanding and harness observations for their benefit, such as anticipating others’ behaviors, rehearsing them under different conditions, and imagining behavior not seen before. Possessing the four inference capabilities is essential for observational learning, but robots do not fully support them and require quality inputs to render inferences feasible.
To explore the viability of robots analyzing others’ interactions in natural conditions, in this dissertation, we focus on formalizing human observational learning and then challenge and evaluate its potential, such as inferring hand behavior from everyday activities. The proof of principle comprises the identification of a formalism covering core capabilities of human observational learning, the instantiation of a framework serving as the object of study, the specification of a scenario that challenges its potential, the verification of proper functioning, and the utility determining inferences remain meaningful. The results show inferences can operate outside the formalism’s functional design despite atypical conditions and breaking a foundational assumption. Moreover, under such conditions, inferences manage to find causal relationships which happen to be meaningful. By introducing this proof of principle and value, we know that robots equipped with the inference formalism operating outside the functional design do not necessarily fail and could provide valuable inferences.