TY - CHAP U1 - Buchbeitrag A1 - Iossifidis, Ioannis A1 - Lawitzky, Gisbert A1 - Knoop, Steffen A1 - Zöllner, Raoul T1 - Towards Benchmarking of Domestic Robotic Assistants T2 - Advances in Human-Robot-Interaction N2 - As service robotics research advances rapidly, availability of objective, reproducible test specifications and evaluation criteria and also of benchmarking is more and more felt to be desirable in the community. As a first step towards benchmarking, in this paper we propose a formalization of tests - exemplified for domestic grasp&place tasks. The underlying philosophy of our approach is to confront the robot system in a black-box manner with requirements of a “rational customer”, and characterize the performance of the system in an objective way by the outcomes of a test-suite tailored to this scenario. A formalized single test description consists of a clear and reproducible specification of the robot’s task and the full context on the one hand, and a number of figures which objectively characterize the test result on the other hand. We illustrate this methodology for the domestic assistance scenario. Y1 - 2004 UR - https://link.springer.com/chapter/10.1007/978-3-540-31509-4_33 SP - 403 EP - 414 PB - Springer Press ER -