MMFSL: A Novel Multimodal Few-Shot Learning Framework for Fault Diagnosis of Industrial Bearings

Unbalanced data with very few samples for special abnormal conditions frequently occur in actual production processes, which can make accurate monitoring of the process state challenging. This article proposes a multimodal few-shot learning method (MMFSL) within a fault diagnosis framework for unbalanced data modeling of industrial bearings. MMFSL can handle two modes of data and therefore contains two data generation channels. The first channel deals with time-series data and the second deals with images. The two modes of generated data are then evaluated using the MMFSL evaluation modules, image evaluation index (IEI), and time-series evaluation index (TEI), to guarantee the quality of the generated data. In addition, the time-series data are analyzed in the time-frequency domain to ensure the consistency of the frequency distribution. Original unbalanced and insufficient data samples are expanded so that the fault diagnosis model can be trained adequately. Through experiments using data from Case Western Reserve University (CWRU) and Pardborn University (PU), generated data by using the MMFSL model can increase fault detection accuracy from 80.9% to 97.5%. At the same time, false alarm rate can be reduced significantly from 4.6% to 0.1%. Moreover, fault classification model accuracy can reach 98.1% compared to 95.4% for the original time-series dataset, and 99.3% compared with 96.5% for the image data extension. Various visualizations of comparison results are also provided to show the reliability of the MMFSL framework.