Robust Pose Estimation via Hybrid Point and Twin Line Reprojection for RGB-D Vision Navigation

Pose estimation is a crucial technique to achieve robot localization using natural features for an RGB-D vision navigation system. However, it still suffers from the impoverishment of robustness and accuracy in complex workspaces. A robust real-time pose estimation approach using hybrid point and twin line reprojection is proposed to allow accurate localization for a vision-based robot under various challenging scenarios, including low texture, low structure, occlusion, and viewpoint and illumination changes. First, an improved adaptive threshold (IAT)-based oriented fast and rotated brief (ORB) feature detection method is developed to extract sufficient IAT-ORB point features that are matched by a multistage
${K}$
-nearest neighbor (KNN)-ratio-test criterion (RTC)-cosine similarity (CS)-random sample consensus (RANSAC) (KRCR) refinement feature matching strategy to generate reliable point matches. Then, a robust EDLines line feature matching method is developed by integrating dynamic matching (DM) and median absolute deviation (MAD) to obtain fine line matches. Furthermore, the hybrid point reprojection errors are built from the fusion of 3-D–2-D and 3-D–3-D point pairs, and the twin line reprojection errors are constructed by integrating virtual right-eye lines generated by depth measurement, which both are incorporated into a novel unified errors’ optimization model to achieve optimal motion estimation by minimizing the reprojection errors of points and lines concurrently. Finally, the extensive experiments verify the robustness, accuracy, and real-time performance of the proposed approach on the public datasets and our real indoor scenarios. The quantitative results demonstrate that our proposed approach gains more accurate and stable localization than other state-of-the-art methods under challenging environments.