Needle Segmentation Using GAN: Restoring Thin Instrument Visibility in Robotic Ultrasound

Ultrasound-guided percutaneous needle insertion is a standard procedure employed in both biopsy and ablation in clinical practices. However, due to the complex interaction between tissue and instrument, the needle may deviate from the in-plane view, resulting in a lack of close monitoring of the percutaneous needle. To address this challenge, we introduce a robot-assisted ultrasound (US) imaging system designed to seamlessly monitor the insertion process and autonomously restore the visibility of the inserted instrument when misalignment happens. To this end, the adversarial structure is presented to encourage the generation of segmentation masks that align consistently with the ground truth in high-order space. This study also systematically investigates the effects on segmentation performance by exploring various training loss functions and their combinations. When misalignment between the probe and the percutaneous needle is detected, the robot is triggered to perform transverse searching to optimize the positional and rotational adjustment to restore needle visibility. The experimental results on ex-vivo porcine samples demonstrate that the proposed method can precisely segment the percutaneous needle (with a tip error of $0.37\pm 0.29$ mm and an angle error of $1.19\pm 0.29$ °). Furthermore, the needle appearance can be successfully restored under the repositioned probe pose in all 45 trials, with repositioning errors of $1.51\pm 0.95~\text {mm}$ and $1.25\pm 0.79$ °.

I. Introduction

Precise needle placement is a crucial procedure in many minimally invasive interventions, such as tissue biopsy and cancerous tissue ablation. Given its radiation-free nature and real-time capabilities, medical ultrasound (US) has been widely used in providing live images to guide needle insertion, thereby enhancing targeting accuracy [1], [2]. However, it is challenging to maintain the needle inside the traditional 2-D US plane during the needle insertion. Certain training and effort are needed for clinical experts to coordinate the alignment between the US probe and the inserted needle, particularly considering US images often suffer from speckle noise, artifacts, and low contrast. Therefore, precise needle segmentation and tracking is a vital and essential step toward accurate needle placement for both biopsy and ablation therapy, particularly for robotic US systems (RUSSs) [3], [4], [5], [6], [7], [8], [9], [10], [11], [12].