Robot Task Primitive Segmentation from Demonstrations Using Only Built-in Kinematic State and Force-Torque Sensor Data

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

Abstract

This paper presents a method for segmentation and classification of kinesthetic demonstrations of robot peg-in-hole tasks using a deep neural network. The presented method depends only on sensor data that is readily available on collaborative robots (kinematic state and forcetorque readings) and does not need any additional sensors. Our method can be used to automatically derive program structures from a single demonstration of a robot task. This can reduce programming time, and make it easier to revise sections of a larger task. We introduce a combined architecture consisting of a Convolutional Neural Network block for raw feature extraction and a Long Short-Term Memory block for tracking the time-evolution of these features. We also extend the model from binary insertion segmentation to multi-class segmentation including pick up, place, free air motions, insertions, and extraction. Through an ablation study and comparison to previous work, we show that the new model performs better on the test set, containing one demonstrator, but significantly better on a generalization set consisting of ten previously unseen demonstrators, reaching an overall accuracy of 70%, which is 15 percentage points better than the closest-performing other method.

Original languageEnglish
Title of host publication2023 IEEE 19th International Conference on Automation Science and Engineering (CASE)
Number of pages7
PublisherIEEE Computer Society
Publication date2023
ISBN (Electronic)9798350320695
DOIs
Publication statusPublished - 2023
Event19th IEEE International Conference on Automation Science and Engineering, CASE 2023 - Auckland, New Zealand
Duration: 26. Aug 202330. Aug 2023

Conference

Conference19th IEEE International Conference on Automation Science and Engineering, CASE 2023
Country/TerritoryNew Zealand
CityAuckland
Period26/08/202330/08/2023
SponsorBeckhoff, CTEK - Combined Technologies
SeriesIEEE International Conference on Automation Science and Engineering
Volume2023-August
ISSN2161-8070

Fingerprint

Dive into the research topics of 'Robot Task Primitive Segmentation from Demonstrations Using Only Built-in Kinematic State and Force-Torque Sensor Data'. Together they form a unique fingerprint.

Cite this