Simulating Virtual Environment and Experience for Training, Exergaming, and Edutainment in eXtended Reality (XR): A Survey

In recent years, eXtended Reality (XR) technologies, including Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR), have emerged as transformative technologies in various domains and have revolutionized the fields of training, exergaming, and edutainment. These immersive XR technologies offer unique opportunities to simulate virtual environments and experiences that facilitate interactive learning and engaging physical activities. The simulation of virtual environments and experiences plays a crucial role in delivering high-quality and effective XR programs. This survey explores existing research and advancements in simulating virtual environments and experiences for training, exergaming, and edutainment in XR. This survey aims at providing an overview of the current state of research and development, discusses the challenges and opportunities in virtual environment and experience simulation, and highlights their future directions. By clarifying the advancements in virtual environment and experience simulation, this survey helps researchers discover the potential of XR in enhancing training, promoting physical activity, and revolutionizing education.

I. Introduction

As an umbrella term, eXtended Reality (XR) [4] is the one that encompasses various immersive technologies, including Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). Figure 1 is a teaser that offers a glimpse into the world of eXtended Reality (XR) technologies, each of these XR technologies has its unique characteristics and applications, showcasing the vast potential for creating captivating and transformative experiences for users. Virtual Reality (VR) [5] refers to a simulated environment that is completely computer-generated. It immerses users in a virtual world, isolating them from their physical surroundings. VR typically involves wearing a head-mounted display (HMD) [6] or using projection systems to give users a sense of presence and immersion within the virtual environment. Users can interact with the virtual world using specialized input devices, such as handheld controllers [7], [8] or motion-tracking systems. Augmented Reality (AR) [9]–[11] overlays digital content onto the real-world environment, blending virtual elements with real-time sensory input. AR applications can be experienced through mobile devices, smart glasses, or headsets equipped with transparent displays. AR enhances the user’s perception of reality [12] by providing additional information, context, or interactive elements in real time. It enables users to interact with both virtual and physical objects, often using gestures or voice commands. Mixed Reality (MR) [13] combines elements of both VR and AR, allowing digital objects to interact with the real world and vice versa. MR seamlessly blends virtual content with the user’s physical environment, creating a cohesive and interactive experience [14]. Users can see and interact with virtual objects while maintaining awareness of their real-world surroundings. MR systems often employ advanced sensors, cameras, and spatial mapping technologies [13], [15] to enable real-time spatial understanding and accurate alignment of virtual objects with the physical environment.