Free-hand gesture recognition with 3D-CNNs for in-car infotainment control in real-time

  • In this contribution we present a novel approach to transform data from time-of-flight (ToF) sensors to be interpretable by Convolutional Neural Networks (CNNs). As ToF data tends to be overly noisy depending on various factors such as illumination, reflection coefficient and distance, the need for a robust algorithmic approach becomes evident. By spanning a three-dimensional grid of fixed size around each point cloud we are able to transform three-dimensional input to become processable by CNNs. This simple and effective neighborhood-preserving methodology demonstrates that CNNs are indeed able to extract the relevant information and learn a set of filters, enabling them to differentiate a complex set of ten different gestures obtained from 20 different individuals and containing 600.000 samples overall. Our 20-fold cross-validation shows the generalization performance of the network, achieving an accuracy of up to 98.5% on validation sets comprising 20.000 data samples. The real-time applicability of our system is demonstrated via an interactive validation on an infotainment system running with up to 40fps on an iPad in the vehicle interior.

Export metadata

Additional Services

Share in Twitter Search Google Scholar
Author:Fabian Sachara, Thomas Kopinski, Alexander Gepperth, Uwe Handmann
Parent Title (English):IEEE Intelligent Transportation Systems Conference (ITSC2017)
Place of publication:Yokohama, Japan
Document Type:Conference Proceeding
Year of Completion:2017
Release Date:2019/04/12
First Page:959
Last Page:964
Zugriff aus dem Hochschulnetz der Hochschule Ruhr West möglich
Institutes:Fachbereich 1 - Institut Informatik
DDC class:600 Technik, Medizin, angewandte Wissenschaften / 620 Ingenieurwissenschaften und Maschinenbau
000 Allgemeines, Informatik, Informationswissenschaft / 004 Informatik
Licence (German):License LogoNo Creative Commons