Affordance estimation for vision-based object replacement on a humanoid robot

Wail Mustafa, Mirko Wächter, Sandor Szedmak, Alejandro Agostini, Dirk Kraft, Tamim Asfour, Justus Piater, Florentin Wörgötter, Norbert Krüger

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review


In this paper, we address the problem of finding replacements of missing objects, involved in the execution of manipulation tasks. Our approach is based on estimating functional affordances for the unknown objects in order to propose replacements. We use a vision-based affordance estimation system utilizing object-wise global features and a multi-label learning method. This method also associates confidence values to the estimated affordances. We evaluate our approach on kitchen-related manipulation affordances. The evaluation also includes testing different scenarios for training the system using large-scale datasets. The results indicate that the system is able to successfully predict the affordances of novel objects. We also implement our system on a humanoid robot and demonstrate the affordance estimation in a real scene.

Original languageEnglish
Title of host publicationProceedings of the 47th International Symposium on Robotics
Publication date2016
ISBN (Print)978-3-8007-4231-8
Publication statusPublished - 2016
Event47th International Symposium on Robotics - Munich, Germany
Duration: 21. Jun 201622. Jun 2016
Conference number: 47


Conference47th International Symposium on Robotics
SponsorABB Robotics, FANUC Deutschland GmbH, KUKA Roboter GmbH, SCHUNK GmbH and Co. KG


Dive into the research topics of 'Affordance estimation for vision-based object replacement on a humanoid robot'. Together they form a unique fingerprint.

Cite this