A Simultaneous Polyp and Lumen Detection Framework Toward Autonomous Robotic Colonoscopy

Colorectal Cancer is one of the deadliest diseases with a high incidence and mortality worldwide. Robotic colonoscopes have been extensively developed to provide alternative solutions for colon screening. Nevertheless, most robotic colonoscopes remain a low autonomy level, which leads to non-intuitive manipulation and limits their clinical translation. This paper proposes a deep learning-based framework for simultaneous polyp and lumen detection, which aims to automates robotic colonoscopes to achieve intelligent and autonomous manipulations in two aspects of navigation and diagnosis. Two fully annotated datasets, including a real colon dataset, with 40186 images, and a colon phantom dataset, are developed to facilitate polyp and lumen detection in both clinical and laboratory environments. Benchmarking of various object detection models achieve an average precision of 0.827, and an average recall of 0.866. Experimental validation is conducted in both a commercialized colon phantom and an ex-vivo porcine colon using an electromagnetically actuated soft-tethered colonoscope as a case study, with the results indicate that the colonoscope can successfully perform autonomous navigation and automatic polyp detection under the proposed unified framework. This work promotes clinical applications of robotic colonoscopy by enhancing its autonomy level with artificial intelligence techniques.

I. Introduction

Coloreactal cancer (CRC), as the third most frequent malignancy and second leading cause of cancer death in the world in 2020 [1], is also one of the most preventable caner. Typically, CRC begins with a tissue growth termed ‘polyp’, which grows slowly in the colon or rectum. By detecting and removing the polyps at an early state, the five-year survival rate of CRC can reach up to 90%, which, however, drastically drops to less than 15% in case of late diagnosis [2]. Therefore, mass and regular colon screening program is strongly recommended for reducing the incidence and mortality of CRC. Conventional colonoscopy represents the ‘gold standard’ for colon screening and treatment [3], which inserts a flexible endoscope (FE) into the patient’s colon to look for and remove suspected pre-cancerous polyps. Typically, three major tasks are involved in the colonoscopy procedure, including (i) navigation of the endoscope through the whole colon, (ii) diagnosis of colon abnormalities (e.g., polyps), and (iii) therapeutic operations (e.g., polypectomy). The duration of conventional colonoscopy is typically 30 to 60 minutes, which, however, may lead to significant discomfort and even pain to patients due to the forcible insertion of the standard FE through the tortuous colon. In addition, it requires extremely high skills for surgeons to manipulate the flexible endoscope due to the high flexibility of the colon and endoscope. These two limitations substantially lower the social acceptance and availability of mass and regular colon screening program [4].