MultiLevel Feature Pooling Network for Uncalibrated Stereo Rectification in Autonomous Vehicles

Stereo vision systems in autonomous vehicles can come with camera sensors with different models and lenses to detect objects at long and short ranges. The intrinsic/extrinsic parameters of the camera sensors, such as the focal length and distortion parameters used for calibration, are not provided. In this article, we present a novel rectification method for uncalibrated stereo cameras based on their corresponding feature points. To obtain reliable key/feature points and descriptors from stereo images, we propose a deep neural network, called a multilevel feature pooling network. The model processes images individually to learn both local and semiglobal features, from which the location and descriptor of each pixel in the image are estimated. We also introduce a method to balance the zoom effect caused by the different focal lengths of the camera lenses and rectify the images by estimating homography matrices for stereo images based on their refined corresponding feature points.