Automated Detection of Non-Informative Frames for Colonoscopy Through a Combination of Deep Learning and Feature Extraction

Colonoscopy is a standard medical examination used to inspect the mucosal surface and detect abnormalities of the colon. Objective assessment and scoring of disease features in the colon are important in conditions such as colorectal cancer and inflammatory bowel disease. However, subjectivity in human disease assessment and measurement is hampered by interobserver variation and several biases. A computer-aided system for colonoscopy video analysis could facilitate diagnosis and disease severity measurement, which would aid in treatment selection and clinical outcome prediction. However, a large number of images captured during colonoscopy are non-informative, making detecting and removing those frames an important first step in performing automated analysis. In this paper, we present a combination of deep learning and conventional feature extraction to distinguish non-informative from informative images in patients with ulcerative colitis. Our result shows that the combination of bottleneck features in the RGB color space and hand-crafted features in the HSV color space can boost the classification performance. Our proposed method was validated using 5-fold cross-validation and achieved an average AUC of 0.939 and an average F1 score of 0.775.