{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T21:52:46Z","timestamp":1775253166021,"version":"3.50.1"},"reference-count":54,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2018,10,18]],"date-time":"2018-10-18T00:00:00Z","timestamp":1539820800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["U1604158"],"award-info":[{"award-number":["U1604158"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["U1509203"],"award-info":[{"award-number":["U1509203"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61751304"],"award-info":[{"award-number":["61751304"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61673160"],"award-info":[{"award-number":["61673160"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Rotating machinery usually suffers from a type of fault, where the fault feature extracted in the frequency domain is significant, while the fault feature extracted in the time domain is insignificant. For this type of fault, a deep learning-based fault diagnosis method developed in the frequency domain can reach high accuracy performance without real-time performance, whereas a deep learning-based fault diagnosis method developed in the time domain obtains real-time diagnosis with lower diagnosis accuracy. In this paper, a multimodal feature fusion-based deep learning method for accurate and real-time online diagnosis of rotating machinery is proposed. The proposed method can directly extract the potential frequency of abnormal features involved in the time domain data. Firstly, multimodal features corresponding to the original data, the slope data, and the curvature data are firstly extracted by three separate deep neural networks. Then, a multimodal feature fusion is developed to obtain a new fused feature that can characterize the potential frequency feature involved in the time domain data. Lastly, the fused new feature is used as the input of the Softmax classifier to achieve a real-time online diagnosis result from the frequency-type fault data. A simulation experiment and a case study of the bearing fault diagnosis confirm the high efficiency of the method proposed in this paper.<\/jats:p>","DOI":"10.3390\/s18103521","type":"journal-article","created":{"date-parts":[[2018,10,18]],"date-time":"2018-10-18T10:55:41Z","timestamp":1539860141000},"page":"3521","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":32,"title":["A Multimodal Feature Fusion-Based Deep Learning Method for Online Fault Diagnosis of Rotating Machinery"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3592-9664","authenticated-orcid":false,"given":"Funa","family":"Zhou","sequence":"first","affiliation":[{"name":"School of Computer and Information Engineering, Henan University, Kaifeng 475004, China"}]},{"given":"Po","family":"Hu","sequence":"additional","affiliation":[{"name":"School of Computer and Information Engineering, Henan University, Kaifeng 475004, China"}]},{"given":"Shuai","family":"Yang","sequence":"additional","affiliation":[{"name":"School of Computer and Information Engineering, Henan University, Kaifeng 475004, China"}]},{"given":"Chenglin","family":"Wen","sequence":"additional","affiliation":[{"name":"School of Automatic, Hangzhou Dianzi University, Hangzhou 310018, China"}]}],"member":"1968","published-online":{"date-parts":[[2018,10,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"439","DOI":"10.1016\/j.measurement.2016.01.023","article-title":"Wheel-bearing fault diagnosis of trains using empirical wavelet transform","volume":"82","author":"Cao","year":"2016","journal-title":"Measurement"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1593","DOI":"10.1016\/j.eswa.2007.08.072","article-title":"A new approach to intelligent fault diagnosis of rotating machinery","volume":"35","author":"Lei","year":"2008","journal-title":"Expert. Syst. Appl."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"65","DOI":"10.3901\/JME.2016.09.065","article-title":"Induction motor fault diagnosis based on deep neural network of sparse auto-encoder","volume":"52","author":"Sun","year":"2016","journal-title":"J. Mech. Eng."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2439","DOI":"10.21595\/jve.2017.17146","article-title":"Research on intelligent fault diagnosis of mechanical equipment based on sparse deep neural networks","volume":"19","author":"Qin","year":"2017","journal-title":"J. Vibroeng."},{"key":"ref_5","first-page":"511","article-title":"Review on operation state assessment and prognostics for mechanical equipment based on hidden markov model","volume":"3","author":"Ji","year":"2017","journal-title":"J. Mech. Strength"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"96","DOI":"10.3901\/CJME.2014.1103.166","article-title":"Feature extraction and recognition for rolling element bearing fault utilizing short-time fourier transform and non-negative matrix factorization","volume":"28","author":"Gao","year":"2015","journal-title":"Chin. J. Mech. Eng."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"467","DOI":"10.1016\/j.ymssp.2008.03.007","article-title":"Diagnosis of subharmonic faults of large roating machinery based on EMD","volume":"23","author":"Wu","year":"2009","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_8","first-page":"1993","article-title":"A data-driven fault propagation analysis method","volume":"8","author":"Zhou","year":"2010","journal-title":"J. Chem. Ind. Eng."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1846","DOI":"10.1109\/TIE.2014.2361115","article-title":"Induction machine bearing fault detection by means of statistical processing of the stray flux measurement","volume":"62","author":"Frosini","year":"2015","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1016\/j.jsv.2017.03.037","article-title":"Time-varying demodulation analysis for rolling bearing fault diagnosis under variable speed conditions","volume":"400","author":"Feng","year":"2017","journal-title":"J. Sound Vib."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Yi, C., Lv, Y., Ge, M., Xiao, H., and Yu, X. (2017). Tensor Singular Spectrum Decomposition Algorithm Based on Permutation Entropy for Rolling Bearing Fault Diagnosis. Entropy, 19.","DOI":"10.3390\/e19040139"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Wang, Y., He, Z., and Zi, Y. (2010). A comparative study on the local mean decomposition and empirical mode decomposition and their applications to rotating machinery health diagnosis. J. Vib. Acoust.","DOI":"10.1115\/1.4000770"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"318","DOI":"10.1016\/j.matcom.2007.12.004","article-title":"Bearing condition monitoring based on shock pulse method and improved redundant lifting scheme","volume":"79","author":"Li","year":"2008","journal-title":"Math. Comput. Simul."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"8096","DOI":"10.3390\/s140508096","article-title":"Wayside bearing fault diagnosis based on a data-driven doppler effect eliminator and transient model analysis","volume":"14","author":"Liu","year":"2014","journal-title":"Sensors"},{"key":"ref_15","first-page":"87","article-title":"The fault detection and diagnosis in rolling element bearings using frequency band entropy","volume":"27","author":"Liu","year":"2012","journal-title":"J. Mech. Eng. Sci."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1295","DOI":"10.3390\/s140101295","article-title":"A One-Versus-All Class Binarization Strategy for Bearing Diagnostics of Concurrent Defects","volume":"14","author":"Ng","year":"2014","journal-title":"Sensors"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"241","DOI":"10.3390\/s100100241","article-title":"A Multi-Fault Diagnosis Method for Sensor Systems Based on Principle Component Analysis","volume":"10","author":"Zhu","year":"2010","journal-title":"Sensors"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Jiang, L., Li, Q., Cui, J., and Xi, J. (2015, January 23\u201325). Rolling bearing fault classification based on higher-order cumulants and BP neural network. Proceedings of the 27th Chinese Control and Decision Conference (2015 CCDC), Qingdao, China.","DOI":"10.1109\/CCDC.2015.7162374"},{"key":"ref_19","first-page":"37","article-title":"Present Situation and Prospect of Data-driven Based Fault Diagnosis Technique","volume":"44","author":"Zhang","year":"2017","journal-title":"Comput. Sci."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"5990","DOI":"10.1109\/TIE.2017.2774777","article-title":"A New Convolutional Neural Network Based Data-Driven Fault Diagnosis Method","volume":"65","author":"Wen","year":"2018","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"134","DOI":"10.1016\/j.conengprac.2017.09.021","article-title":"Data-driven root-cause fault diagnosis for multivariate non-linear processes","volume":"70","author":"Rashidi","year":"2017","journal-title":"Control Eng. Pract."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"527","DOI":"10.1049\/joe.2017.0190","article-title":"Fault diagnosis using kernel principal component analysis for hot strip mill","volume":"2017","author":"Zhang","year":"2017","journal-title":"J. Eng."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1016\/j.eswa.2006.04.020","article-title":"Application of nonlinear feature extraction and support vector machines for fault diagnosis of induction motors","volume":"33","author":"Widodo","year":"2007","journal-title":"Expert Syst. Appl."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Zhou, S., Qian, S., Chang, W., Xiao, Y., and Cheng, Y. (2018). A Novel Bearing Multi-Fault Diagnosis Approach Based on Weighted Permutation Entropy and an Improved SVM Ensemble Classifier. Sensors, 18.","DOI":"10.3390\/s18061934"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"5627","DOI":"10.3390\/s150305627","article-title":"An SVM-Based Solution for Fault Detection in Wind Turbines","volume":"15","author":"Santos","year":"2015","journal-title":"Sensors"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Jiang, Q., Shen, Y., Li, H., and Xu, F. (2018). New Fault Recognition Method for Rotary Machinery Based on Information Entropy and a Probabilistic Neural Network. Sensors, 18.","DOI":"10.3390\/s18020337"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Li, Y., Cheng, G., Pang, Y., and Kuai, M. (2018). Planetary Gear Fault Diagnosis via Feature Image Extraction Based on Multi Central Frequencies and Vibration Signal Frequency Spectrum. Sensors, 18.","DOI":"10.3390\/s18061735"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Liu, C., Cheng, G., Chen, X., and Pang, Y. (2018). Planetary gears feature extraction and fault diagnosis method based on VMD and CNN. Sensors, 18.","DOI":"10.3390\/s18051523"},{"key":"ref_29","first-page":"387","article-title":"Application of wavelet packet in motor fault diagnosis","volume":"34","author":"Wang","year":"2013","journal-title":"J. Changchun Univ. Technol."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Sohaib, M., Kim, C.-H., and Kim, J.-M. (2017). A Hybrid Feature Model and Deep-Learning-Based Bearing Fault Diagnosis. Sensors, 17.","DOI":"10.3390\/s17122876"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Wu, Z., Guo, Y., Lin, W., Yu, S., and Ji, Y. (2018). A Weighted Deep Representation Learning Model for Imbalanced Fault Diagnosis in Cyber-Physical Systems. Sensors, 18.","DOI":"10.3390\/s18041096"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Li, S., Liu, G., Tang, X., Lu, J., and Hu, J. (2017). An Ensemble Deep Convolutional Neural Network Model with Improved D-S Evidence Fusion for Bearing Fault Diagnosis. Sensors, 17.","DOI":"10.3390\/s17081729"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Li, C., S\u00e1nchez, R.-V., Zurita, G., Cerrada, M., and Cabrera, D. (2016). Fault Diagnosis for Rotating Machinery Using Vibration Measurement Deep Statistical Feature Learning. Sensors, 16.","DOI":"10.3390\/s16060895"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"15221","DOI":"10.3390\/s131115221","article-title":"A New Approach for Improving Reliability of Personal Navigation Devices under Harsh GNSS Signal Conditions","volume":"13","author":"Dhital","year":"2013","journal-title":"Sensors"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Jing, L., Wang, T., Zhao, M., and Wang, P. (2017). An Adaptive Multi-Sensor Data Fusion Method Based on Deep Convolutional Neural Networks for Fault Diagnosis of Planetary Gearbox. Sensors, 17.","DOI":"10.3390\/s17020414"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Zhang, R., Peng, Z., Wu, L., Yao, B., and Guan, Y. (2017). Fault Diagnosis from Raw Sensor Data Using Deep Neural Networks Considering Temporal Coherence. Sensors, 17.","DOI":"10.3390\/s17030549"},{"key":"ref_37","first-page":"521","article-title":"Deep learning","volume":"436","author":"Lecun","year":"2015","journal-title":"Nature"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"504","DOI":"10.1126\/science.1127647","article-title":"Reducing the Dimensionality of Data with Neural Networks","volume":"313","author":"Hinton","year":"2006","journal-title":"Science"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1016\/j.sigpro.2016.07.028","article-title":"Fault diagnosis of rotary machinery components using a stacked denoising autoencoder-based health state identification","volume":"130","author":"Lu","year":"2017","journal-title":"Signal Process."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1016\/j.ymssp.2015.10.025","article-title":"Deep neural networks: A promising tool for fault characteristic mining and intelligent classification of rotating machinery with massive data","volume":"72","author":"Jia","year":"2016","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1016\/j.ymssp.2015.11.014","article-title":"Construction of hierarchical diagnosis network based on deep learning and its application in the fault pattern recognition of rolling element bearings","volume":"72\u201373","author":"Gan","year":"2016","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/j.neucom.2015.06.008","article-title":"Multimodal deep support vector classification with homologous features and its application to gearbox fault diagnosis","volume":"168","author":"Li","year":"2015","journal-title":"Neurocomputing"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"4973","DOI":"10.1109\/TIE.2017.2767540","article-title":"Lifting Net A Novel Deep Learning Network with Layerwise Feature Learning from Noisy Mechanical Data for Fault Classification","volume":"65","author":"Pan","year":"2018","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"4290","DOI":"10.1109\/TIE.2017.2762639","article-title":"Deep Residual Networks with Dynamically Weighted Wavelet Coefficients for Fault Diagnosis of Planetary Gearboxes","volume":"65","author":"Zhao","year":"2018","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_45","first-page":"60","article-title":"A fully automatic onine mode identiflcation method for multi-mode processes","volume":"42","author":"Zhang","year":"2016","journal-title":"Acta Autom. Sin."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1285","DOI":"10.1016\/S0888-3270(03)00088-8","article-title":"Cyclostationary modelling of rotating machine vibration signals","volume":"18","author":"Antoni","year":"2004","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_47","first-page":"015","article-title":"Comparison studies on sampling method of rotating machine vibration signals in time and angular domain","volume":"1","author":"Liao","year":"2007","journal-title":"J. Changsha Commun. Univ."},{"key":"ref_48","first-page":"38","article-title":"Implementation of Rotating Machine Vibration Signals Acquisition System","volume":"4","author":"Du","year":"2004","journal-title":"Instrum. Technol."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"188","DOI":"10.1504\/IJDATS.2014.062458","article-title":"A comparative study of sound and vibration signals in detection of rotating machine faults using support vector machine and independent component analysis","volume":"6","author":"Saimurugan","year":"2014","journal-title":"Int. J. Data Anal. Tech. Strat."},{"key":"ref_50","first-page":"80","article-title":"Kalman Filtering and Fault Diagnosis of Rotating Machines Vibration Signal","volume":"39","author":"Zhao","year":"2014","journal-title":"Instrum. Tech. Sens."},{"key":"ref_51","first-page":"4","article-title":"Wavelet signal filtering and extracting characteristic of rotating machines vibration signal","volume":"28","author":"Dai","year":"2004","journal-title":"Inf. Technol."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1016\/j.microrel.2017.03.006","article-title":"Deep neural networks-based rolling bearing fault diagnosis","volume":"75","author":"Chen","year":"2017","journal-title":"Microelectron. Reliabil."},{"key":"ref_53","first-page":"364","article-title":"Short Time Zero Crossing Rate of Vibration Signal and Self-Organizing Map for Bearing Faults detection and Diagnosis","volume":"65","author":"Haroun","year":"2017","journal-title":"Int. Conf. Autom. Control Telecommun. Signals"},{"key":"ref_54","unstructured":"(2018, May 10). Bearing Data Centre, Western Reserve University. Available online: http:\/\/csegroups.case.edu\/bearingdatacenter\/home."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/10\/3521\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T20:38:12Z","timestamp":1775248692000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/10\/3521"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,10,18]]},"references-count":54,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2018,10]]}},"alternative-id":["s18103521"],"URL":"https:\/\/doi.org\/10.3390\/s18103521","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,10,18]]}}}