Deep Learning for Tumor Localization with Depth Estimation: A Minimally Invasive Robotics-Assisted Approach

This work is aimed at deploying deep learning models to characterize tissue stiffness properties during telerobotic palpation and localization of tissue abnormality while estimating its depth. The method relies on using a minimally invasive probe with a rigidly mounted tactile sensor at the tip to capture the force distribution map and the indentation depth for each tactile element, thereby generating a stiffness map for the palpated tissue. The probe is attached to a Mitsubishi PA10 robot with hybrid impedance control architecture controlled by a haptic device which enables the user to telerobatically palpate the remote tissue and semi-autonomously obtain the required information from the tissue. When data are collected, a convolutional neural network (CNN) is utilized for tumor classification and an artificial neural network (ANN) is used to estimate the depth of the tumor within the tissue. The proposed method is verified by classifying tumor phantoms using CNN with an accuracy of 95.24% for stiffness and using ANN with an accuracy of 82.14% for depth.