Q-learning-based navigation for mobile robots in continuous and dynamic environments

Abderraouf Maoudj*, Anders Lyhne Christensen

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

Abstract

Autonomous collision-free navigation in dynamic environments is essential in many mobile robot applications. Reinforcement learning has the potential to automate the control design process and it has been successfully applied to path planning and mobile robot navigation. Obtaining effective navigation strategies with reinforcement learning is, however, still very time-consuming, especially in complex continuous environments where the robot can easily get trapped. In this paper, we propose a novel state space definition that includes information about the robot's most recent action. The proposed state variables for the target and nearby obstacles are binary and denote presence (or absence) within a corresponding region in the robot's frame of reference. In addition, we propose two heuristic algorithms that provide the robot with basic prior knowledge about a promising action in each state, which reduces the initial time-consuming blind exploration and thereby significantly shortens training time. We train a robot using our improved Q-learning approach to navigate in continuous environments in a high-fidelity simulator. In a series of experiments, we demonstrate the effectiveness of the proposed approach in terms of training time and solution quality compared to state-of-the-art approaches.

Original languageEnglish
Title of host publication2021 IEEE 17th International Conference on Automation Science and Engineering, CASE 2021
PublisherIEEE
Publication date2021
Pages1338-1345
ISBN (Electronic)9781665418737
DOIs
Publication statusPublished - 2021
Event17th IEEE International Conference on Automation Science and Engineering, CASE 2021 - Lyon, France
Duration: 23. Aug 202127. Aug 2021

Conference

Conference17th IEEE International Conference on Automation Science and Engineering, CASE 2021
Country/TerritoryFrance
CityLyon
Period23/08/202127/08/2021
SeriesIEEE International Conference on Automation Science and Engineering
Volume2021-August
ISSN2161-8070

Bibliographical note

Publisher Copyright:
© 2021 IEEE.

Fingerprint

Dive into the research topics of 'Q-learning-based navigation for mobile robots in continuous and dynamic environments'. Together they form a unique fingerprint.

Cite this