Classification of visual interest based on gaze and facial features for human-robot interaction

It is important for a social robot to know if a nearby human is showing interest in interacting with it. We approximate this interest with expressed visual interest. To find it, we train a number of classifiers with previously labeled data. The input features for these are facial features like head orientation, eye gaze and facial action units, which are provided by the OpenFace library. As training data, we use video footage collected during an in-the-wild human-robot interaction scenario, where a social robot was approaching people at a cafeteria to serve them water. The most successful classifier that we trained tested at a 94% accuracy for detecting interest on an unrelated testing dataset. This allows us to create an effective tool for our social robot, which enables it to start talking to people only when it is fairly certain that the addressed persons are interested in talking to it.