Selection Techniques for 3D Extended Desktop Workstation with AR HMD

Extending a standard desktop workstation (i.e. a screen, a mouse, a keyboard) with virtual scenes displayed on an Augmented Reality Head-Mounted Display (AR HMD) offers many identified advantages including limited physical space requirements, very large and flexible display spaces, and 3D stereoscopic views. While the technologies become more mainstream, the remaining open question is how to interact with such hybrid workstations that combine 2D views displayed on a physical monitor and 3D views displayed on a HoloLens. For a selection task, we compared mouse-based interaction (standard for 2D desktop workstations) and direct touch interaction in mid-air (standard for 3D AR) while considering different positions of the 3D scene according to a physical monitor. To extend mouse-based selection to 3D views, we experimentally explored different interaction metaphors where the mouse cursor moves either on a horizontal or a vertical plane in a 3D virtual scene. To check for ecological validity of our results, we conducted an additional study focusing on interaction with a 2D/3D Gapminder dataset visualization. The results show 1) that the mouse-based interaction, as compared to direct touch interaction in mid-air, is easy and efficient, 2) that using a vertical plane placed in front of the 3D virtual scene to mimic the double screen metaphor outperforms other interaction techniques and 3) that flexibility is required to allow users to choose the selection techniques and to position the 3D virtual scene relative to the physical monitor. Based on these results, we derive interaction design guidelines for hybrid workstations.