3-D RoI-Aware U-Net for Accurate and Efficient Colorectal Tumor Segmentation

Segmentation of colorectal cancerous regions from 3-D magnetic resonance (MR) images is a crucial procedure for radiotherapy. Automatic delineation from 3-D whole volumes is in urgent demand yet very challenging. Drawbacks of existing deep-learning-based methods for this task are two-fold: 1) extensive graphics processing unit (GPU) memory footprint of 3-D tensor limits the trainable volume size, shrinks effective receptive field, and therefore, degrades speed and segmentation performance and 2) in-region segmentation methods supported by region-of-interest (RoI) detection are either blind to global contexts, detail richness compromising, or too expensive for 3-D tasks. To tackle these drawbacks, we propose a novel encoder–decoder-based framework for 3-D whole volume segmentation, referred to as 3-D RoI-aware U-Net (3-D RU-Net). 3-D RU-Net fully utilizes the global contexts covering large effective receptive fields. Specifically, the proposed model consists of a global image encoder for global understanding-based RoI localization, and a local region decoder that operates on pyramid-shaped in-region global features, which is GPU memory efficient and thereby enables training and prediction with large 3-D whole volumes. To facilitate the global-to-local learning procedure and enhance contour detail richness, we designed a dice-based multitask hybrid loss function. The efficiency of the proposed framework enables an extensive model ensemble for further performance gain at acceptable extra computational costs. Over a dataset of 64 T2-weighted MR images, the experimental results of four-fold cross-validation show that our method achieved 75.5% dice similarity coefficient (DSC) in 0.61 s per volume on a GPU, which significantly outperforms competing methods in terms of accuracy and efficiency. The code is publicly available.

I. Introduction

Colorectal cancer strikes more than 1.4 million people and accounts for 694 000 deaths globally in 2012 [1]. It is more common in developed countries, for example, in the USA, colorectal cancer is the second leading cause of cancer-related mortalities [2]. In the current clinical routine of radiotherapy, due to the advantages of magnetic resonance (MR) imaging for soft tissue enhancement [3], colorectal cancer regions are manually recognized and delineated from volumetric MR images acquired for treatment, including surgery and radiation therapy. However, this procedure is laborious, time consuming, and observer dependent, thus suffers from the tedious effort and limited reproducibility. Therefore, automatic colorectal tumor detection and segmentation methods are highly demanded to improve the clinical routine.