3D Slicer-AR-Bridge: 3D Slicer AR Connection for Medical Image Visualization and Interaction with AR-HMD

In recent years, increasing interest has been revealed in medical Augmented Reality (AR), particularly in medical education, preoperative planning, and intra-operative guidance. However, there lacks a platform to effectively integrate each essential part of surgical AR, including visualization, interaction, tracking, and medical image processing, hindering new beginners or surgeons from easily building up any comprehensive AR applications. Therefore, we introduce 3D Slicer-AR-Bridge, a novel framework taking advantage of open-source medical image processing software 3D Slicer and advanced AR developing toolkits, to enable fast generation of surgical AR applications with high performance, high extensibility, and transparent data flow. The bridge is first built upon a seamless medical data sharing interface supporting volumes, segmentations, models, and annotations, between 3D Slicer and Unity, together with corresponding visualization and interaction methods. Sensor resources from AR devices are then synchronized to the bridge during visualization using holographic remoting to enable in-situ display with high render quality. Meanwhile, all resources are highly accessible in 3D Slicer, while a general user interface is provided to operate all the elements, enabling the creation of an in-situ AR application with little coding, which we believe will greatly benefit the community and clinical research. The performance tests in our paper indicate that 3D Slicer-AR-Bridge can do data exchange efficiently. At the end of our paper, we provide two different applications built with the bridge with little modification to the codes, to exemplify how the framework can help the researchers in surgical AR.