A Standalone Augmented Reality Tool for X-ray-like Vision: Harnessing Body Tracking and Sensor Capabilities of HoloLens 2

Augmented Reality can enable "x-ray-like vision" for detailed spatial knowledge about patient anatomy, making it a useful tool for medical training. Prior applications have thus far relied on external cameras tethered to a powerful processing unit to track a patient’s body and overlay anatomy. This paper presents a first-of-its-kind standalone AR application that relies solely on the sensors of the Microsoft HoloLens 2 and performs all processing in real time in the cloud. We measured 3D body pose tracking alignment accuracy by measuring how closely 3D virtual anatomy renderings aligned with real anatomical landmarks. Across multiple landmarks and body poses, the median 3D body pose tracking alignment was 8.7 cm (IQR = 6.9 cm). User feedback underscored the application’s potential for teaching anatomy, visualizing trauma cases, and enhancing training simulations.