Volltext-Downloads (blau) und Frontdoor-Views (grau)

Fusion of Audio- and Visual Cues for Real-Life Emotional Human Robot Interaction

  • Recognition of emotions from multimodal cues is of basic interest for the design of many adaptive interfaces in human-machine interaction (HMI) in general and human-robot interaction (HRI) in particular. It provides a means to incorporate non-verbal feedback in the course of interaction. Humans express their emotional and affective state rather unconsciously exploiting their different natural communication modalities such as body language, facial expression and prosodic intonation. In order to achieve applicability in realistic HRI settings, we develop person-independent affective models. In this paper, we present a study on multimodal recognition of emotions from such auditive and visual cues for interaction interfaces. We recognize six classes of basic emotions plus the neutral one of talking persons. The focus hereby lies on the simultaneous online visual and accoustic analysis of speaking faces. A probabilistic decision level fusion scheme based on Bayesian networks is applied to draw benefit of the complementary information from both – the acoustic and the visual – cues. We compare the performance of our state of the art recognition systems for separate modalities to the improved results after applying our fusion scheme on both DaFEx database and a real-life data that captured directly from robot. We furthermore discuss the results with regard to the theoretical background and future applications.

Export metadata

Additional Services

Share in Twitter Search Google Scholar


Author:Ahmad Rabie, Uwe Handmann
Parent Title (English):Pattern Recognition
Document Type:Conference Proceeding
Year of Completion:2011
Date of first Publication:2019/03/08
Creating Corporation:Joint Pattern Recognition Symposium
Release Date:2019/03/08
First Page:346
Last Page:355
Institutes:Fachbereich 1 - Institut Informatik
DDC class:000 Allgemeines, Informatik, Informationswissenschaft / 004 Informatik
Licence (German):License LogoNo Creative Commons