Pedestrian localization based on image plane constraint

Human-robot safe operation requires pedestrian localization and tracking capability from a mobile robot, so a novel method for tracking walking humans on a mobile robot which installs a 2D LiDAR (Light Detection and Ranging) and a camera is proposed. Firstly, the 2D LiDAR point-to-image plane constraint is established after the camera and 2D LiDAR are calibrated with external parameters, and the horizontal 2D LiDAR beams taken from approximately the knee level are visible from the camera image. Secondly, pedestrians are tracked by MobileNet-SSD network and SORT algorithm, the 2D LiDAR data are obtained from the tracking bounding box through the point-to-image constraint plane. Thirdly, the adaptive parametric DBSCAN clustering algorithm is designed for classifying the point cloud as clusters. If two clusters satisfy predefined conditions defined as two legs, a human target is generated and tracked. Finally, hazard assessment is obtained by pedestrian velocity calculated through the tracking result. Results comparison among different methods demonstrates the proposed method competitive performance for the pedestrian localization task.