Collaborative Perceptual Anchoring Across Static, Wearable and Robotic Sensors

Service robots are increasingly becoming a common part of our everyday lives. Beside manufacturing, they can assists us in many tasks, including cleaning, lawn-moving, and serving drinks. One of the challenges that robots face is building and maintaining a symbolic world model of their environment, which enables the robot to communicate with a user about objects and to plan tasks. The process of securing the correct correspondence between the symbols in the world model and the objects in the physical world is called perceptual anchoring. Several works on perceptual anchoring have proposed solutions to this challenge, using data from robot-mounted sensors. This presents a challenge due to the fact that the robot can only sense its immediate surroundings. This motivates another approach, where other sensors, which are not on the robot, are used to achieve collaborative sensing of the environment. This approach comes with its own set of challenges, which the current state of the art has yet to fully explore and address. This PhD project address the challenges of collaborative sensing across Internetof-things (IoT), wearable and robotics sensors. The dissertation presents four technical contributions. Firstly, a comparison of methods for data association for IoT and robot sensors. Secondly, a framework for collaborative anchoring using IoT and robot sensors, for manipulation tasks. Thirdly, a complete anchoring pipeline for robot and wearable sensors. And finally, a review of the technical innovations and lessons learned in the FacilityCobot project applying the contributions in practice. In conclusion, this PhD project provides a foundation for further advancements in the field of collaborative sensing across IoT, wearable and robotic sensors.