Globally Learnable Point Set Registration Between 3D CT and Multi-view 2D X-ray Images of Hip Phantom

2D-3D registration is a crucial step in Image-Guided Intervention, such as spine surgery, total hip re-placement, and kinematic analysis. To find the information in common between pre-operative 3D CT images and intra-operative X-ray 2D images is vital to plan and navigate. In a nutshell, the goal is to find the movement and rotation of the 3D body’s volume to make them reorient with the patient body in the 2D image space. Due to the loss of dimensionality and different sources of images, efficient and fast registration is challenging. To this end, we propose a novel approach to incorporate a point set Neural Network to combine the information from different views, which enjoys the robustness of the traditional method and the geometrical information extraction ability. The pre-trained Deep BlindPnP captures the global information and local connectivity, and each implementation of view-independent Deep BlindPnP in different view pairs will select top-priority pairs candidates. The transformation of different viewpoints into the same coordinate will accumulate the correspondence. Finally, a POSEST-based module will output the final 6 DoF pose. Extensive experiments on a real-world clinical dataset show the effectiveness of the proposed framework compared to the single view. The accuracy and computation speed are improved by incorporating the point set neural network.