Reinforcement Learning-based Autonomous Sensor Control via Simultaneous Learning of Policies and State-Action Spaces

Reinforcement learning is a promising candidate methodology for achieving situational awareness across an area of interest by controlling and processing data acquired from a multi-site, multi-modality sensing grid. We previously reported successful detection, tracking, classification, and identification of objects operating within the area of interest monitored by a sensing grid. We combined online kernel least squares policy iteration (an online reinforcement learning method combining dictionary learning with classical Q-learning) with a particle tracker to achieve these results. The work reported here extends these prior results to show that sensor fusion allows our online reinforcement learning methodology to successfully control a multi-modality sensor platform (consisting of a pan/tilt/zoom electro-optical camera, a radar, and passive radio frequency sensor) to maintain persistent surveillance of objects of interest. We evaluate our sensor fusion and online reinforcement learning methodology through Gazebo simulation of a realistic test and experimentation location. Our results demonstrate that policies based on our methodology trained on processed simulated sensor data perform as well as policies trained on known ground truth data. These results also show that the learned policies offer significant generalization ability, with the sensor platform being able to successfully track an observed object well past the observed training period. Moreover, we show that our reinforcement learning methodology shows promising results in dealing with the data association problem. These results further build towards our ultimate goal of achieving automated situational awareness across a heterogeneous sensor grid.