Cut & Recombine: Reuse of Robot Action Components Based on Simple Language Instructions

Minija Tamosiunaite, Mohamad Javad Aein, Jan-Matthias Braun, Tomas Kulvicius, Irena Markievicz, Jurgita Kapociute-Dzikiene, Rita Valteryte, Andrei Haidu, Dimitrios-Chrysostomos Chrysostomou, Barry Bridge, Thomas Krilavicius, Daiva Vitkute-Adzgauskiene, Michael Beetz, Ole Madsen, Aleš Ude, Norbert Krüger, Florentin Wörgötter

Research output: Contribution to journalJournal articleResearchpeer-review

176 Downloads (Pure)

Abstract

Human beings can generalize from one action to similar ones. Robots cannot do this and progress concerning information transfer between robotic actions is slow. We have designed a system that performs action generalization for manipulation actions in different scenarios. It relies on an action representation for which we perform code-snippet replacement, combining information from different actions to form new ones. The system interprets human instructions via a parser using simplified language. It uses action and object names to index action data tables (ADTs), where execution-relevant information is stored. We have created an ADT database from three different sources (KUKA LWR, UR5, and simulation) and show how a new ADT is generated by cutting and recombining data from existing ADTs. To achieve this, a small set of action templates is used. After parsing a new instruction, index-based searching finds similar ADTs in the database. Then the action template of the new action is matched against the information in the similar ADTs. Code snippets are extracted and ranked according to matching quality. The new ADT is created by concatenating code snippets from best matches. For execution, only coordinate transforms are needed to account for the poses of the objects in the new scene. The system was evaluated, without additional error correction, using 45 unknown objects in 81 new action executions, with 80% success. We then extended the method including more detailed shape information, which further reduced errors. This demonstrates that cut & recombine is a viable approach for action generalization in service robotic applications.

Original languageEnglish
JournalThe International Journal of Robotics Research
Volume38
Issue number10-11
Pages (from-to)1179-1207
ISSN0278-3649
DOIs
Publication statusPublished - 1. Sep 2019

Keywords

  • Cognitive robotics
  • control architectures and programming
  • manipulation
  • manipulation planning
  • service robotics

Fingerprint Dive into the research topics of 'Cut & Recombine: Reuse of Robot Action Components Based on Simple Language Instructions'. Together they form a unique fingerprint.

Cite this