TransVAT: Transformer Encoder with Variational Attention for Few-Shot Fault Diagnosis

Fault diagnosis plays a critical role in ensuring safety and minimizing downtime across various industries. However, due to the difficulties in acquiring fault signals in practical engineering systems, labeled samples are often scarce. To address this issue, few-shot learning has emerged as a promising approach for bearing fault diagnosis in recent years. Recent studies with promising results have demonstrated the effectiveness of using Transformer and variational attention in this field. Compared to conventional methods, the Transformer has demonstrated superior performance in feature extraction and classification. Variational attention, on the other hand, permits a distribution of attention weights and enhances the interpretability of models. This method can identify pertinent features and offer perceptions of the root causes of faults. Therefore, the proposed model, TransVAT, is based on the relation network of few-shot learning and replaces the dot-product attention in the Transformer encoder with variational attention for feature extraction. The experimental findings demonstrate that the model performs well with limited data, especially on the one-shot task.