Autonomous Neurosurgical Instrument Segmentation Using End-To-End Learning

Monitoring surgical instruments is an essential task in computer-assisted interventions and surgical robotics. It is also important for navigation, data analysis, skill assessment and surgical workflow analysis in conventional surgery. However, there are no standard datasets and benchmarks for tool identification in neurosurgery. To this end, we are releasing a novel neurosurgical instrument segmentation dataset called NeuroID for advancing research in the field. Delineating surgical tools from the background requires accurate pixel-wise instrument segmentation. In this paper, we present a comparison between three encoder-decoder approaches to binary segmentation of neurosurgical instruments, where we classify each pixel in the image to be either tool or background. A baseline performance was obtained by using heuristics to combine extracted features. We also extend the analysis to a publicly available robotic instrument segmentation dataset and include its results. The source code for our methods and the neurosurgical instrument dataset will be made publicly available (http://brl.ee.washington.edu/robotics/surgical-robotics/neurosurgical-instrument-segmentation) to facilitate reproducibility.

1. Introduction

Automated instrument identification from surgical video frames will aid the development of intelligent applications in instrument tracking, pose estimation, augmented reality overlay, visual servoing, analysis of surgical phase, and will aid development of safety mechanisms for surgery. In addition to benefiting different components in surgery, the position information of instruments also helps in measuring the context-awareness of a surgeon, potentially helping to reduce human errors. Bounding box recognition of instruments is not sufficient due to the coarse boundaries generated around the regions of interest. The dense prediction of tool versus background through semantic segmentation enhances safety by parsing the whole scene without suppressing occluded or slanted objects.