Large-Scale Person Detection and Localization using Overhead Fisheye Cameras

Location determination finds wide applications in daily life. Instead of existing efforts devoted to localizing tourist photos captured by perspective cameras, in this article, we focus on devising person positioning solutions using overhead fisheye cameras. Such solutions are advantageous in large field of view (FOV), low cost, anti-occlusion, and unaggressive work mode (without the necessity of cameras carried by persons). However, related studies are quite scarce, due to the paucity of data. To stimulate research in this exciting area, we present LOAF, the first large-scale overhead fisheye dataset for person detection and localization. LOAF is built with many essential features, e.g., i) the data cover abundant diversities in scenes, human pose, density, and location; ii) it contains currently the largest number of annotated pedestrian, i.e., 457K bounding boxes with groundtruth location information; iii) the body-boxes are labeled as radius-aligned so as to fully address the positioning challenge. To approach localization, we build a fisheye person detection network, which exploits the fisheye distortions by a rotation-equivariant training strategy and predict radius-aligned human boxes end-to-end. Then, the actual locations of the detected persons are calculated by a numerical solution on the fisheye model and camera altitude data. Extensive experiments on LOAF validate the superiority of our fisheye detector w.r.t. previous methods, and show that our whole fisheye positioning solution is able to locate all persons in FOV with an accuracy of0.5 m, within 0.1 s.

1. Introduction

Accurate position finding of persons attracts growing interest from both research and industrial communities, since it plays a crucial role in numerous location-sensitive application scenarios (e.g., surveillance, smart home, public health). Nevertheless, due to the line-of-sight (LOS) issue, GPS is unreliable in interior spaces and urban canyon. To overcome such limitation, various alternative solutions are investigated. Signal based solutions, including Bluetooth [13] and Wi-Fi [74], are popular, but they are easily interfered by changing environments and nearby human bodies [73]. A complementary stream of work is vision based; they typically make use of traditional cameras, RGBD cameras, or in-built smartphone cameras, and enjoy the advantage of reliable services. To get location information, visual positioning solutions usually refer to a pre-acquired 3D map or a geo-tagged database as the scene representation [63], or directly utilize the captured image to estimate the camera pose [39].