Integrating View Magnification into an Augmented Reality Head-Mounted Display to Support Surgery

Medical applications are a good use case for augmented reality (AR) technology as the overlay of virtual content on patients supports the visualization of body components, pre-surgery scans and real-time biometric data. Thus, there is increasing use of AR for medical training, surgery planning and tele-mentoring with growing access to affordable AR hardware. Also AR is now being used in actual surgeries as the technology can provide hands free interaction and line-of-sight rendering of virtual content to support surgeons. However, the magnification of surgery sites is often needed and AR hardware cameras are typically only used for environmental tracking or the real-time recording/streaming of the current user’s view. Access to a dedicated magnification view is not readily available from commercial AR headsets. This paper identifies several challenges to adding an external camera to provide magnification functionality to be integrated into an AR scene. A proposed system is described with an overview of ongoing and future work.