Virtual reality providing ground truth for machine learning and programming by demonstration

In this contribution, it is demonstrated how the development, parameterization and evaluation of an intelligent cognitive system can be supported by means of Virtual Reality (VR). Based on new approaches in object and action recognition, the consortium in the EU funded project IntellAct1 is developing such an automated system for the intelligent understanding of manual tasks and robot-based manipulations. In this context, the VR setup of the Institute for Man-Machine Interaction (MMI) is able to deliver ideal data, such as detailed positions and joint data of objects and kinematics involved. In addition, it allows for the synthesis of distinct and error-free data, such as collisions, bounding boxes and contact events. This way, Virtual Reality serves as a reference and source for "ground truth" data for designing, training and benchmarking the intelligent cognitive system during its development and parameterization. Furthermore this approach enables descriptive and secure ways for the visualization of results in the targeted environments, such as the highly automated laboratories on-board the International Space Station (see Fig.1).