A DDPG-Based Method of Autonomous Catheter Navigation in Virtual Environment

Vascular interventional surgery is the main method for treating cardiovascular diseases. But navigating endovascular catheters through the vascular tree is a highly challenging task even for highly trained specialists. Automation of this task can reduce the burden on surgeons and is expected to improve the surgical outcomes. Although there have been relevant studies utilizing reinforcement learning algorithms to realize autonomous navigation of catheter in the virtual environment Cathsim. However, the kinematics model of the catheter in Cathsim does not conform to the operating mode of the catheter in real vascular interventional surgery. Besides, there are problems such as low success rates of catheter autonomous navigation tasks. To address these issues, this paper modifies the kinematics model of the catheter in Cathsim and designs a catheter autonomous navigation model based on reinforcement learning DDPG (Deep Deterministic Policy Gradient) algorithm. The experimental results show that the agents trained through DDPG in this paper performs better than the agents trained through PPO (Proximal Policy optimization) in other studies in terms of navigation task success rate, completion time, and contact force between the catheter and vascular wall during the navigation process.