Learning to Calibrate Straight Lines for Fisheye Image Rectification

This paper presents a new deep-learning based method to simultaneously calibrate the intrinsic parameters of fisheye lens and rectify the distorted images. Assuming that the distorted lines generated by fisheye projection should be straight after rectification, we propose a novel deep neural network to impose explicit geometry constraints onto processes of the fisheye lens calibration and the distorted image rectification. In addition, considering the nonlinearity of distortion distribution in fisheye images, the proposed network fully exploits multi-scale perception to equalize the rectification effects on the whole image. To train and evaluate the proposed model, we also create a new large-scale dataset labeled with corresponding distortion parameters and well-annotated distorted lines. Compared with the state-of-the-art methods, our model achieves the best published rectification quality and the most accurate estimation of distortion parameters on a large set of synthetic and real fisheye images.

1. Introduction

Fisheye cameras have been widely used in many computer vision tasks [31], [34], [9], [16], [2] because of their large field of view (FOV), however, the images taken by fisheye cameras always suffer from severe geometric distortion simultaneously. When processing the geometric toward vision systems equipped with fisheye lens, calibrating the intrinsic parameters is usually the first step we should do to rectify the distorted images.