Visual Goal Human-Robot Communication Framework With Few-Shot Learning: A Case Study in Robot Waiter System

A conventional adopted method for operating a waiter robot is based on the static position control, where predefined goal positions are marked on a map. However, this solution is not optimal in a dynamic setting, such as in a coffee shop or an outdoor catering event, because the customers often change their positions. This article explores an alternative human-robot interface design where a human operator communicates the identity of the customer to the robot instead. Inspired by how human communicates, we propose a framework for communicating a visual goal to the robot, through interactive two-way communications. The framework exploits concepts from two machine learning domains: human-in-the-loop machine learning, where active learning is used to acquire informative data, and deep metric learning, where a suitable embedding can improve the learning ability of a classifier. We also propose novel class imbalance handling techniques, which aim to actively alleviate the class imbalance problem found to be important in this mode of communication. The framework is evaluated using publicly available pedestrian datasets. We demonstrate that the proposed framework can help reduce the number of required two-way interactions and increases the robustness of the predictive model. We successfully implement the framework on a mobile robot for a delivery service in a cafe-like environment. Through the online visual goal human-robot communication, the robot can detect, recognize, and autonomously navigate to the target customer.