Model-Free Perception-Based Control via Q-Learning with an Application to Heat-Seeking Missile Guidance

Modern perception-based sensing schemes incorporate machine learning and high-dimensional image observations to control system states, but face issues of perception error and incomplete dynamics and state information. To address these issues, we propose a novel perception-based control strategy using model-free output feedback Q-learning that incorporates a Faster R-CNN convolutional neural network. We specifically investigate the optimal control problem of a linear time-invariant, discrete-time system given only the observation image data. We evaluate the data-driven control design process in ideal perception and degraded perception conditions. We show that the resulting controller from output feedback Q-learning is non-optimal, but the optimality loss is bounded with bounded perception error. Simulated results on a simple missile, whose seeker head observes synthetic images of the target heat source modeled as a blurry ball of light, show the efficacy of the proposed model-free perception-based control framework.