Using Reinforcement Learning to Attenuate for Stochasticity in Robot Navigation Controllers

James Gillespie, Inaki Rano, Nazmul Siddique, José Santos, Mehdi Khamassi

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

Abstract

Braitenberg vehicles are bio-inspired controllers for sensor-based local navigation of wheeled robots that have been used in multiple real world robotic implementations. The common approach to implement such non-linear control mechanisms is through neural networks connecting sensing to motor action, yet tuning the weights to obtain appropriate closed-loop navigation behaviours can be very challenging. Standard approaches used hand tuned spiking or recurrent neural networks, or learnt the weights of feedforward networks using evolutionary approaches. Recently, Reinforcement Learning has been used to learn neural controllers for simulated Braitenberg vehicle 3a - a bio-inspired model of target seeking for wheeled robots - under the assumption of noiseless sensors. Real sensors, however, are subject to different levels of noise, and multiple works have shown that Braitenberg vehicles work even on outdoor robots, demonstrating that these control mechanisms work in harsh and dynamic environments. This paper shows that a robust neural controller for Braitenberg vehicle 3a can be learnt using policy gradient reinforcement learning in scenarios where sensor noise plays a non negligible role. The learnt controller is robust and tries to attenuate the effects of noise in the closed- loop navigation behaviour of the simulated stochastic vehicle. We compare the neural controller learnt using Reinforcement Learning with a simple hand tuned controller and show how the neural control mechanism outperforms a naïve controller. Results are illustrated through computer simulations of the closed-loop stochastic system.

Original languageEnglish
Title of host publication2019 IEEE Symposium Series on Computational Intelligence (SSCI)
Number of pages10
PublisherIEEE
Publication dateDec 2019
Pages705-713
Article numberhal-02324129f
ISBN (Print)9781728124858
ISBN (Electronic)9781728124858
DOIs
Publication statusPublished - Dec 2019
EventThe 2019 IEEE Symposium Series on Computational Intelligence: SSCI 2019 - Xiamen, China
Duration: 6. Dec 20199. Dec 2019

Conference

ConferenceThe 2019 IEEE Symposium Series on Computational Intelligence
Country/TerritoryChina
CityXiamen
Period06/12/201909/12/2019

Keywords

  • braitenberg vehicle
  • reinforcement learning
  • stochastic environment

Fingerprint

Dive into the research topics of 'Using Reinforcement Learning to Attenuate for Stochasticity in Robot Navigation Controllers'. Together they form a unique fingerprint.

Cite this