A3RT: Attention-Aware AR Teleconferencing with Life-Size 2.5D Video Avatars

Augmented Reality (AR) teleconferencing aims to enable remotely separated users to meet with each other in their own physical spaces as if they are face-to-face. Among all solutions, the video-avatar-based approach has the advantage of balancing fidelity and the sense of co-presence using easy-to-setup devices, including only a camera and an AR Head-Mounted Display (HMD). However, non-verbal cues indicating “who is looking at whom” are always lost or misdelivered in multiparty teleconferencing experiences. To make users aware of such non-verbal cues, existing solutions explore screen-based visualizations, incorporate additional hardware, or alter to use a virtual avatar representation. However, they lack immersion, are less feasible for everyday usage due to complex installations, or lose the fidelity of remote users’ authentic appearances. In this paper, we decompose such attention awareness into the awareness of being looked at and the awareness of attention between other users and address them in a decoupled process. Specifically, through a user study, we first find an unobtrusive and reasonable layout “Attention Circle” to retarget a looker’s head gaze to the one being looked at. We then conduct the second user study to find an effective and intuitive “rotatable 2.5D video avatar with attention thumbnail” visualization to aid users in being aware of other users’ attention. With the design choice distilled from the studies, we implement A3RT, a proof-of-concept prototype system that empowers attention-aware 2.5D-video-avatar-based multiparty AR teleconferencing in an easy, everyday setup. Ablation and usability studies on the prototype verify the effectiveness of our proposed components and the full system.