Soft Sensor-Based Deep Temporal-Graph Convolutional Network for Applications in Human Motion Tracking

The popularity of accurate real-time motion tracking wearables with multiple on-body soft stretchable sensors stems from their advantages of easy wearability, high stretchability, and low cost. Due to its high nonlinearity and hysteresis, challenges remain in terms of the accuracy, calibration, and difficulty in estimating human motion from soft sensor signals. In order to address these issues, we present a novel deep learning-based calibration method for human motion tracking. Combining with temporal convolutional network (TCN), a weighted graph convolutional network (GCN) and the shortcut structure, we propose a deep temporal-graph convolutional network with Pearson correlation coefficient (T-PGCN). The proposed T-PGCN is designed to extract the temporal features by a TCN module and the spatial features by a GCN module, subsequently, by means of a shortcut connection, the outputs and the original input data are fed into a fully connected neural network (FCNN) module to obtain the predicted positions at the tracking points. The effectiveness and generalization of our proposed T-PGCN model are demonstrated using two datasets: the DFM Motion Tracking dataset and the SSG Hand Posing Tracking dataset. The results show that T-PGCN performs better than the baseline models without increasing much training time.