Virtual Model Interaction Based on Single RGB Camera

Nowadays, gesture recognition has been sufficiently developed and used in many areas, for example, you can control the model by your gesture, which can provide the audience with a more intuitive understanding of the displayed model and make it easier for the operator to present. However, most methods are based on specific device such as depth sensor, which contributes to gesture recognition cannot be extensively implemented, using a normal RGB camera to track the hand and recognize the gesture becomes a significant issue. Lightweight is also a problem, real-time hand detection and tracking is feasible due to the low level of lightweight. Under the circumstance, cost saving of gesture recognition is a significant issue to exploit. We present a new method using a real-time on-device hand tracking solution to control virtual models in Unreal Engine. We have introduced OneEuro Filter to handle the jitter that occurs when the hand is in motion, affecting the control of the model.