A Vision-Based Guidance Algorithm for Entering Buildings Through Windows for Delivery Drones

This article introduces a novel vision-based guidance algorithm to make a camera-equipped drone capable of collision-free entrance to a building through its windows. To eliminate the reliance on any extra sensor (ultrasonic range finders, lidars, depth sensors, stereo cameras, etc.), which are frequently used in the literature, we introduce an image-based optimization scheme to detect the exact boundaries of a window portal section. The optimization is based on a common facade segmentation technique, referred to as projected profiles, and relies on the assumption that the window portal is one of the least light-reflective segments in a facade structure. In combination with a visual tracking algorithm, our method elevates the tracking performance to avoid any guidance failures due to tracking drifts. As a result, the drone gets a rich extent of navigation data during the flight enabling it to reliably enter through the window. The window portal detection is evaluated using two online facade datasets, suggesting a 75% success rate in finding the correct entrance passage. Also, it is shown that the visual tracking performance is improved up to 58% in the specific case of tracking an entrance portal. The efficacy and real-time capability of the overall vehicle guidance are experimentally demonstrated using a rotorcraft micro aerial vehicle.