Robust Uncalibrated Rectification With Low Geometric Distortion Under Unbalanced Field of View Circumstances

Rectification is a standard process in every system that requires multiviews. Existing algorithms largely work on similar field of view (FoV) cases where the two views are mostly identical. Dissimilarities between different FoVs can generate unexpected errors during the optimization process, resulting in a large amount of rectification errors and unwanted geometric distortion. In this study, we present a full pipeline to rectify uncalibrated images captured by cameras that have dissimilar FoVs under the constraints of geometric distortion. The proposed method contains two main parts: Field of View Neutralization and Rectification with Adaptive Geometric Constraints. The Field of View Neutralization module estimates the transformation matrix to compensate for the imbalance between different views. In addition, this module improves the overall quality of correspondences by removing misleading feature matching pairs. Finally, by applying an adaptive optimization process with the geometric constraints involved, our method addresses the overdistortion issue while maintaining small rectification errors. Extensive experiments are conducted to demonstrate the robust performance of the proposed method. Besides the existing datasets, we provide our dissimilar FoVs dataset with multiple baselines to examine the performance. Our method outperforms the existing algorithms in terms of both rectification errors and geometric distortion rates.