Design of human-computer interaction system based on virtual reality and its application in the dissemination of study lodge culture

Virtual reality (VR) is an emerging computer environment that utilizes virtual environments to achieve virtual assembly. It has broad application prospects and important value in multiple fields. Interactivity is one of the main characteristics of virtual environments. In order to improve the intelligence of the cultural experience process in the "study lodge" and meet the dissemination of traditional Chinese culture among a wider audience, VR technology is introduced to design and apply an intelligent human-computer interaction system for the "study lodge" from both hardware and software aspects. Firstly, the hardware structure of the system was designed and MEMS measurement and communication equipment was determined. Secondly, with the support of hardware devices, VR technology is matched with PC terminals to establish an indoor 3D environment on the terminal devices. Communication between different servers and terminals is established to achieve the web embedding and "study lodge" communication control in 3D scenes. Using touch screen interaction as the interactive end of the human-computer interaction system, we adopt the method of developing human-computer interaction interfaces to design good interaction between multiple terminals. The user interface interaction function and virtual handwriting function were implemented in the Unity engine. Finally, the features of the collected data were extracted through human-computer interaction method, and the extracted features were input into the classifier for feature recognition, achieving human-computer interaction.

I. Introduction

The three basic characteristics of virtual reality (VR) technology are immersion, interactivity, and imagination, among which interactivity refers to the way users interact with the virtual world through specific means of communication. The implementation of VR technology cannot be achieved without the support of hardware devices. There are two main types of VR devices. The first type is mobile head display devices, such as Gear VR, which need to be plugged into a mobile phone for use. The second type is external head display devices, such as HTC Vive, which need to be used by an external computer. The interaction mode of these two types of VR devices is basically achieved through operating handles. There are three main ways to achieve interaction between humans and virtual scenes: one is virtual interaction, which is controlled through virtual buttons in the virtual scene; The second is physical interaction, which utilizes physical devices such as control handles to achieve interaction with virtual scenes; The third is direct interaction. Through data gloves or Gesture recognition, different action commands are made to control objects in the virtual scene. In VR, most human-computer interaction methods are achieved through virtual or physical interaction. With the continuous development of VR devices and human-computer interaction, direct interaction, especially gesture interaction, has become a more friendly and natural interaction method in VR. Gesture recognition is the precondition of gesture interaction. At present, Gesture recognition is mainly realized by two ways, one is Gesture recognition method based on sensors. Taking Gesture recognition and VR human-computer interaction as research objects, Li et al. [1] proposed and designed a new prototype system for assisting VR devices in human-computer interaction. The second is vision-based Gesture recognition, which uses ordinary cameras to collect data at a low cost, and uses machine learning or neural networks to recognize gesture images. Su et al. [2] proposed a human-computer interaction method for Gesture recognition based on improved YOLOv3. Vision-based deep learning Gesture recognition method is a hotspot in current research, but using deep learning requires providing many gesture sample data for training. When there are only a few samples, deep learning is prone to over fitting.