Autonomous Navigation via a Deep Q Network with One-Hot Image Encoding

Contributing USMA Research Unit(s)

Electrical Engineering and Computer Science, Robotics Research Center

Publication Date

Fall 9-19-2019

Publication Title

2019 IEEE International Symposium on Measurement and Control in Robotics

Document Type

Conference Proceeding


Common autonomous driving techniques employ various combinations of convolutional and deep neural networks to safely and efficiently navigate unique road and traffic conditions. This paper investigates the feasibility of employing a reinforcement learning (RL) model for autonomous navigation using a low dimensional input. While many navigation applications generate each individual state as a function of a frame's raw pixel information, we use a deep Q network (DQN) with reduced input dimensionality to train a mobile robot to continuously remain within a lane around an elliptical track. We accomplish this by using a one-hot encoding scheme that assigns a binary variable to each element in a square array. This value is a function of whether the input frame detects the presence of a lane boundary. Our ultimate goal was to determine the minimum number of training samples required to consistently train the robot to complete one cycle around the track, from multiple starting positions and directions, without crossing a lane boundary. We found that by intelligently balancing exploration and exploitation of its environment, as well as the rewards for staying in the lane, the robot was able to achieve its goal with a small number of samples.

Record links to items hosted by external providers may require fee for full-text.