Deep Multimodal Learning and Fusion Based Intelligent Fault Diagnosis Approach

• 作者: Huifang Li Jianghang Huang Jingwei Huang Senchun Chai Leilei Zhao Yuanqing Xia
• 来源期刊: 北京理工大学学报（英文版）
• 年，卷（期）: 2021,（2）
• 页码范围: 172-185
• 页数: 14页
• 语种: 英文
• DOI: 10.15918/j.jbit1004-0579.2021.017

摘要

Industrial Internet of Things (IoT) connecting society and industrial systems represents a tremendous and promising paradigm shift. With IoT, multimodal and heterogeneous data from in-dustrial devices can be easily collected, and further analyzed to discover device maintenance and health related potential knowledge behind. IoT data-based fault diagnosis for industrial devices is very helpful to the sustainability and applicability of an IoT ecosystem. But how to efficiently use and fuse this multimodal heterogeneous data to realize intelligent fault diagnosis is still a challenge. In this paper, a novel Deep Multimodal Learning and Fusion (DMLF) based fault diagnosis meth-od is proposed for addressing heterogeneous data from IoT environments where industrial devices coexist. First, a DMLF model is designed by combining a Convolution Neural Network (CNN) and Stacked Denoising Autoencoder (SDAE) together to capture more comprehensive fault knowledge and extract features from different modal data. Second, these multimodal features are seamlessly in-tegrated at a fusion layer and the resulting fused features are further used to train a classifier for re-cognizing potential faults. Third, a two-stage training algorithm is proposed by combining super-vised pre-training and fine-tuning to simplify the training process for deep structure models. A series of experiments are conducted over multimodal heterogeneous data from a gear device to veri-fy our proposed fault diagnosis method. The experimental results show that our method outper-forms the benchmarking ones in fault diagnosis accuracy.