Having a clear understanding of the user behaviour is essential to elicit effective and relevant coaching strategies. One of the main tasks of the Council of Coaches, related to sensing and profiling, aims at developing a Holistic Behaviour Analysis Framework (HBAF) that will serve as the principal generator of knowledge on the user’s behaviours. This knowledge will provide relevant background information for improving the user-coach interaction and supporting the dialog and argumentation process with objective and continuous behaviour-related contents.
A Glimpse of the HBAF
The HBAF is responsible for sensing, identifying and quantifying human behaviour in order to nurture the dialogues, arguments and interactions between users and virtual coaches. The HBAF analyzes raw multimodal data and extracts useful information regarding the user’s behaviour. In particular, four principal components of human behaviour, namely physical, social, emotional and cognitive, are analyzed over different periods of time (short and long-term behaviours). The detected behaviours are constantly pushed to the Knowledge Base platform through a secured connection for further analysis. Finally, the provided information can be used in order to feed the dialogues among coaches. In Figure 1, we present the initial design of the HBAF architecture.
Traditional vs Virtual Sensing
The project moves beyond current trends in automatic behaviour analysis while approaching the problem in a holistic fashion, thus modelling, inferring and combining the distinct domains of behaviour including physical, emotional, cognitive and social aspects. Apart from traditional multimodal sensing techniques such as accelerometers, GPS or video, the coaches will be particularly exploited as a sort of virtual sensors to gather relevant behaviour data during the conversational interaction with users.
These virtual sensors will allow to extract relevant behaviour information that cannot be measured through traditional sensing such as memory loss or task engagement. Furthermore, advanced machine learning and data mining methods will be developed to mine and detect relevant behaviour changes, which may in turn demand an intervention from the coaches. In this regard, this project goes beyond prior work while not only focusing on the detection of a change in any of the domains of behaviour but also on the quantification of the significance of the detected changes and their nature or root cause.