Robotics Framework for Object Tracking using FPGA with Novel Automatic Image Labelling

Autonomous robots require the ability to perceive their environment. This must be done in a power-efficient manner to allow them to operate for an extended duration of time. Convolutional neural networks (CNN) are typically used to process image data but they require large amounts of processing power to deploy. CNNs can be efficiently implemented on an FPGA achieving low power consumption. In this work, we present a framework for implementing CNNs on an MPSoC that can be used in robotics applications. A method for automatic image labelling is used to create a dataset for training the neural network. The model is trained using TensorFlow and the weights are automatically exported and programmed onto the FPGA. An example application is developed to showcase the proposed framework. The application achieves a 428% increase in performance and a 432% increase in power efficiency when using hardware acceleration compared to running the application on a CPU.