PhysMViT: Non-contact facial video physiological signal measurement network based on multiscale vision transformer

Remote photoplethysmography (rPPG) can use optical sensors such as cameras to capture the periodicity of skin color caused by heartbeats, and further calculate physiological signals by algorithms. In the future, rPPG technology will be widely applied on the mobile terminal, which has huge potential value. This paper designs an end-to-end heart rate signal detection network PhysMViT based on multi-scale Transformer, and builds a pyramid structure in the transformer to mine different scale information of facial video sequence feature maps. Subsequently, the designed network PhysMViT was trained on two public datasets UBFC and MMPD, and the training effect was verified. The experimental results show that the PhysMViT network has achieved good accuracy in all indicators on the dataset UBFC, and the indicators on the MMPD have also exceeded most networks. Finally, ablation experiments are carried out to verify the effectiveness of the design of each part of the PhysMViT network.