Content based Surgical Video Retrieval via Multi-Deep Features Fusion

Medicine has seen a rapid increase in the number of minimally invasive surgeries that benefit patients in various ways due to economic and clinical factors. In most hospitals such surgeries are often recorded and stored. The use of these recordings includes many potential applications like for medical education, patient disease analysis, surgical error analysis and skill assessment, etc. However, manual search in this database of long-term videos is exceptionally laborious and time-consuming. Therefore, there is a critical need for an efficient and effective content-based video search system for surgical videos. On the other hand, recent progress in deep learning showed the convolutional neural network (CNN) as an effective feature representation learning model. In this paper, we exploit the multiple CNNs for feature representation and proposed a multi feature fusion based approach for Content based Surgical Video Retrieval. To validate our approach we choose Surgical Actions 160 dataset, where the results demonstrate the effectiveness of feature fusion and outperforms the other state-of-the-arts.