An Efficient Gyro-Aided Optical Flow Estimation in Fast Rotations With Auto-Calibration

Optical flow (OF) measures the displacement of the projections of visual landmarks on the image plane and is widely used for motion and structure estimations. Fast rotational motion results in large image motion and poses a major challenge to many OF algorithms. To improve the performance of OF estimations, gyroscopes can be adopted to predict the image motion between adjacent frames when the camera motion is dominated by rotation. This paper first proposes an offline method for calibrating the camera focal length, gyroscope bias, the relative orientation and time offset between the camera and gyroscopes without using a man-made target. Then a simple and efficient method is proposed to aid the OF computation with the gyroscope readings. Our approach is evaluated using a low-cost endoscope-gyroscope system and shows improved visual tracking performance as compared to the stand-alone OF algorithm. The method is suited for use on lightweight platforms equipped with gyroscopes and an endoscopic camera, which potentially can provide efficient image guidance in endoscopic procedures.