Teaching a Robot the Semantics of Assembly Tasks

T. R. Savarimuthu, A. G. Buch, C. Schlette, N. Wantia, Jürgen Roßmann, D. Martinez, G. Alenya, C. Torras, Ales Ude, Bojan Nemec, Aljaz Kramberger, Florentin Worgotter, Eren Erdal Aksoy, Jérémie Papon, Simon Haller, J. Piater, N. Kruger

Research output: Contribution to journalJournal articleResearchpeer-review

165 Downloads (Pure)

Abstract

We present a three-level cognitive system in a learning by demonstration context. The system allows for learning and transfer on the sensorimotor level as well as the planning level. The fundamentally different data structures associated with these two levels are connected by an efficient mid-level representation based on so-called 'semantic event chains.' We describe details of the representations and quantify the effect of the associated learning procedures for each level under different amounts of noise. Moreover, we demonstrate the performance of the overall system by three demonstrations that have been performed at a project review. The described system has a technical readiness level (TRL) of 4, which in an ongoing follow-up project will be raised to TRL 6.
Original languageEnglish
JournalIEEE Transactions on Systems, Man, and Cybernetics: Systems
Volume48
Issue number5
Pages (from-to)670-692
ISSN1083-4427
DOIs
Publication statusPublished - 1. May 2018

Keywords

  • Benchmark testing
  • Context
  • Planning
  • Robot sensing systems
  • Semantics
  • Trajectory
  • Learning by demonstration (LbD)
  • object recognition
  • robotic assembly
  • vision

Cite this

Savarimuthu, T. R. ; Buch, A. G. ; Schlette, C. ; Wantia, N. ; Roßmann, Jürgen ; Martinez, D. ; Alenya, G. ; Torras, C. ; Ude, Ales ; Nemec, Bojan ; Kramberger, Aljaz ; Worgotter, Florentin ; Aksoy, Eren Erdal ; Papon, Jérémie ; Haller, Simon ; Piater, J. ; Kruger, N. / Teaching a Robot the Semantics of Assembly Tasks. In: IEEE Transactions on Systems, Man, and Cybernetics: Systems. 2018 ; Vol. 48, No. 5. pp. 670-692.
@article{e39b439a27824926ae29b88b9b2c6a60,
title = "Teaching a Robot the Semantics of Assembly Tasks",
abstract = "We present a three-level cognitive system in a learning by demonstration context. The system allows for learning and transfer on the sensorimotor level as well as the planning level. The fundamentally different data structures associated with these two levels are connected by an efficient mid-level representation based on so-called 'semantic event chains.' We describe details of the representations and quantify the effect of the associated learning procedures for each level under different amounts of noise. Moreover, we demonstrate the performance of the overall system by three demonstrations that have been performed at a project review. The described system has a technical readiness level (TRL) of 4, which in an ongoing follow-up project will be raised to TRL 6.",
keywords = "Benchmark testing, Context, Planning, Robot sensing systems, Semantics, Trajectory, Learning by demonstration (LbD), object recognition, robotic assembly, vision",
author = "Savarimuthu, {T. R.} and Buch, {A. G.} and C. Schlette and N. Wantia and J{\"u}rgen Ro{\ss}mann and D. Martinez and G. Alenya and C. Torras and Ales Ude and Bojan Nemec and Aljaz Kramberger and Florentin Worgotter and Aksoy, {Eren Erdal} and J{\'e}r{\'e}mie Papon and Simon Haller and J. Piater and N. Kruger",
year = "2018",
month = "5",
day = "1",
doi = "10.1109/TSMC.2016.2635479",
language = "English",
volume = "48",
pages = "670--692",
journal = "I E E E Transactions on Systems, Man and Cybernetics, Part A: Systems & Humans",
issn = "1083-4427",
publisher = "Institute of Electrical and Electronics Engineers",
number = "5",

}

Savarimuthu, TR, Buch, AG, Schlette, C, Wantia, N, Roßmann, J, Martinez, D, Alenya, G, Torras, C, Ude, A, Nemec, B, Kramberger, A, Worgotter, F, Aksoy, EE, Papon, J, Haller, S, Piater, J & Kruger, N 2018, 'Teaching a Robot the Semantics of Assembly Tasks', IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 48, no. 5, pp. 670-692. https://doi.org/10.1109/TSMC.2016.2635479

Teaching a Robot the Semantics of Assembly Tasks. / Savarimuthu, T. R.; Buch, A. G.; Schlette, C.; Wantia, N.; Roßmann, Jürgen ; Martinez, D.; Alenya, G.; Torras, C.; Ude, Ales; Nemec, Bojan; Kramberger, Aljaz; Worgotter, Florentin; Aksoy, Eren Erdal; Papon, Jérémie; Haller, Simon; Piater, J.; Kruger, N.

In: IEEE Transactions on Systems, Man, and Cybernetics: Systems, Vol. 48, No. 5, 01.05.2018, p. 670-692.

Research output: Contribution to journalJournal articleResearchpeer-review

TY - JOUR

T1 - Teaching a Robot the Semantics of Assembly Tasks

AU - Savarimuthu, T. R.

AU - Buch, A. G.

AU - Schlette, C.

AU - Wantia, N.

AU - Roßmann, Jürgen

AU - Martinez, D.

AU - Alenya, G.

AU - Torras, C.

AU - Ude, Ales

AU - Nemec, Bojan

AU - Kramberger, Aljaz

AU - Worgotter, Florentin

AU - Aksoy, Eren Erdal

AU - Papon, Jérémie

AU - Haller, Simon

AU - Piater, J.

AU - Kruger, N.

PY - 2018/5/1

Y1 - 2018/5/1

N2 - We present a three-level cognitive system in a learning by demonstration context. The system allows for learning and transfer on the sensorimotor level as well as the planning level. The fundamentally different data structures associated with these two levels are connected by an efficient mid-level representation based on so-called 'semantic event chains.' We describe details of the representations and quantify the effect of the associated learning procedures for each level under different amounts of noise. Moreover, we demonstrate the performance of the overall system by three demonstrations that have been performed at a project review. The described system has a technical readiness level (TRL) of 4, which in an ongoing follow-up project will be raised to TRL 6.

AB - We present a three-level cognitive system in a learning by demonstration context. The system allows for learning and transfer on the sensorimotor level as well as the planning level. The fundamentally different data structures associated with these two levels are connected by an efficient mid-level representation based on so-called 'semantic event chains.' We describe details of the representations and quantify the effect of the associated learning procedures for each level under different amounts of noise. Moreover, we demonstrate the performance of the overall system by three demonstrations that have been performed at a project review. The described system has a technical readiness level (TRL) of 4, which in an ongoing follow-up project will be raised to TRL 6.

KW - Benchmark testing

KW - Context

KW - Planning

KW - Robot sensing systems

KW - Semantics

KW - Trajectory

KW - Learning by demonstration (LbD)

KW - object recognition

KW - robotic assembly

KW - vision

U2 - 10.1109/TSMC.2016.2635479

DO - 10.1109/TSMC.2016.2635479

M3 - Journal article

VL - 48

SP - 670

EP - 692

JO - I E E E Transactions on Systems, Man and Cybernetics, Part A: Systems & Humans

JF - I E E E Transactions on Systems, Man and Cybernetics, Part A: Systems & Humans

SN - 1083-4427

IS - 5

ER -