Chance-Constrained Control with Imperfect Perception Modules

Autonomous systems are required to operate in different environments, but recognizing the current environment is often challenging. For example, an autonomous vehicle should stop or obey a speed limit according to a traffic sign, but state-of-the-art perception modules (e.g., neural networks) do not guarantee the correctness of their reading of the traffic sign. Considering such uncertain outputs of a perception module, which in effect determines modes, we propose a chance-constrained control formulation that with high probability guarantees the satisfaction of a set of constraints associated with the possible modes. To do this, we present a method based on the Bayes rule and sampling to calculate the probability of each mode. We prove that our approach can ensure satisfying constraints of novel situations, which have not been used during training of the perception module. Also, to account for the error due to limited data, we present a robust formulation that guarantees constraint satisfaction with high confidence. In an autonomous vehicle example, we train a neural network that classifies traffic signs and show that given each output of the neural network, our motion planning approach guarantees the constraint satisfaction with high probability.