{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,15]],"date-time":"2026-11-15T01:54:04Z","timestamp":1794707644811,"version":"4.9.1"},"reference-count":66,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2021,12,22]],"date-time":"2021-12-22T00:00:00Z","timestamp":1640131200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Korea Technology and Information Promotion Agency","award":["S3126818"],"award-info":[{"award-number":["S3126818"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Statistical features extraction from bearing fault signals requires a substantial level of knowledge and domain expertise. Furthermore, existing feature extraction techniques are mostly confined to selective feature extraction methods namely, time-domain, frequency-domain, or time-frequency domain statistical parameters. Vibration signals of bearing fault are highly non-linear and non-stationary making it cumbersome to extract relevant information for existing methodologies. This process even became more complicated when the bearing operates at variable speeds and load conditions. To address these challenges, this study develops an autonomous diagnostic system that combines signal-to-image transformation techniques for multi-domain information with convolutional neural network (CNN)-aided multitask learning (MTL). To address variable operating conditions, a composite color image is created by fusing information from multi-domains, such as the raw time-domain signal, the spectrum of the time-domain signal, and the envelope spectrum of the time-frequency analysis. This 2-D composite image, named multi-domain fusion-based vibration imaging (MDFVI), is highly effective in generating a unique pattern even with variable speeds and loads. Following that, these MDFVI images are fed to the proposed MTL-based CNN architecture to identify faults in variable speed and health conditions concurrently. The proposed method is tested on two benchmark datasets from the bearing experiment. The experimental results suggested that the proposed method outperformed state-of-the-arts in both datasets.<\/jats:p>","DOI":"10.3390\/s22010056","type":"journal-article","created":{"date-parts":[[2021,12,23]],"date-time":"2021-12-23T02:02:57Z","timestamp":1640224977000},"page":"56","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":41,"title":["Bearing Fault Diagnosis Using Multidomain Fusion-Based Vibration Imaging and Multitask Learning"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4578-952X","authenticated-orcid":false,"given":"Md Junayed","family":"Hasan","sequence":"first","affiliation":[{"name":"Department of Electrical, Electronics and Computer Engineering, University of Ulsan, Ulsan 44610, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1823-1304","authenticated-orcid":false,"given":"M. M. Manjurul","family":"Islam","sequence":"additional","affiliation":[{"name":"Department of Computer Science, American International University-Bangladesh, Dhaka 1229, Bangladesh"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5185-1062","authenticated-orcid":false,"given":"Jong-Myon","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of Electrical, Electronics and Computer Engineering, University of Ulsan, Ulsan 44610, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,12,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"107233","DOI":"10.1016\/j.ymssp.2020.107233","article-title":"A new deep auto-encoder method with fusing discriminant information for bearing fault diagnosis","volume":"150","author":"Mao","year":"2021","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"971","DOI":"10.1007\/s10845-020-01600-2","article-title":"Bearing fault diagnosis base on multi-scale CNN and LSTM model","volume":"32","author":"Chen","year":"2021","journal-title":"J. Intell. Manuf."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"138426","DOI":"10.1109\/ACCESS.2019.2937828","article-title":"A multi-stage hybrid fault diagnosis approach for rolling element bearing under various working conditions","volume":"7","author":"Yan","year":"2019","journal-title":"IEEE Access"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Hasan, M.J., Kim, J., Kim, C.H., and Kim, J.-M. (2020). Health State Classification of a Spherical Tank Using a Hybrid Bag of Features and K-Nearest Neighbor. Appl. Sci., 10.","DOI":"10.3390\/app10072525"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1109\/TIM.2019.2903699","article-title":"A new online detection approach for rolling bearing incipient fault via self-adaptive deep feature matching","volume":"69","author":"Mao","year":"2019","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"108002","DOI":"10.1016\/j.measurement.2020.108002","article-title":"A novel health indicator based on the Lyapunov exponent, a probabilistic self-organizing map, and the Gini-Simpson index for calculating the RUL of bearings","volume":"164","author":"Rai","year":"2020","journal-title":"Measurement"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Hasan, M.J., Sohaib, M., and Kim, J.-M. (2020). A Multitask-Aided Transfer Learning-Based Diagnostic Framework for Bearings under Inconsistent Working Conditions. Sensors, 20.","DOI":"10.3390\/s20247205"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"837","DOI":"10.1016\/j.neucom.2015.07.020","article-title":"A novel intelligent method for mechanical fault diagnosis based on dual-tree complex wavelet packet transform and multiple classifier fusion","volume":"171","author":"Qu","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"330","DOI":"10.1016\/j.jsv.2017.03.029","article-title":"Sparse discriminant manifold projections for bearing fault diagnosis","volume":"399","author":"Chen","year":"2017","journal-title":"J. Sound Vib."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1293","DOI":"10.1109\/TIE.2019.2898619","article-title":"Intelligent fault identification based on multisource domain generalization towards actual diagnosis scenario","volume":"67","author":"Zheng","year":"2019","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"3539","DOI":"10.1109\/TIE.2017.2752151","article-title":"Scalable and unsupervised feature engineering using vibration-imaging and deep learning for rotor system diagnosis","volume":"65","author":"Oh","year":"2017","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"110023","DOI":"10.1016\/j.measurement.2021.110023","article-title":"A bearing fault diagnosis method based on multiscale dispersion entropy and GG clustering","volume":"185","author":"Zhang","year":"2021","journal-title":"Measurement"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1016\/j.apacoust.2014.08.016","article-title":"Application of empirical mode decomposition and artificial neural network for automatic bearing fault diagnosis based on vibration signals","volume":"89","author":"Ali","year":"2015","journal-title":"Appl. Acoust."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1347","DOI":"10.1007\/s10033-017-0189-y","article-title":"A deep learning approach for fault diagnosis of induction motors in manufacturing","volume":"30","author":"Shao","year":"2017","journal-title":"Chin. J. Mech. Eng."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"4272","DOI":"10.21595\/jve.2017.18549","article-title":"A hydraulic fault diagnosis method based on sliding-window spectrum feature and deep belief network","volume":"19","author":"Wang","year":"2017","journal-title":"J. Vibroeng."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1016\/j.neucom.2018.05.002","article-title":"A novel optimized SVM classification algorithm with multi-domain feature and its application to fault diagnosis of rolling bearing","volume":"313","author":"Yan","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Sohaib, M., Kim, C.-H., and Kim, J.-M. (2017). A Hybrid Feature Model and Deep-Learning-Based Bearing Fault Diagnosis. Sensors, 17.","DOI":"10.3390\/s17122876"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2648","DOI":"10.1109\/TIM.2019.2928346","article-title":"An enhanced intelligent diagnosis method based on multi-sensor image fusion via improved deep learning network","volume":"69","author":"Wang","year":"2019","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1848","DOI":"10.1109\/ACCESS.2018.2886343","article-title":"Deep decoupling convolutional neural network for intelligent compound fault diagnosis","volume":"7","author":"Huang","year":"2018","journal-title":"IEEE Access"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zhang, W., Peng, G., Li, C., Chen, Y., and Zhang, Z. (2017). A new deep learning model for fault diagnosis with good anti-noise and domain adaptation ability on raw vibration signals. Sensors, 17.","DOI":"10.20944\/preprints201701.0132.v1"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"3299","DOI":"10.1109\/TIE.2016.2527623","article-title":"A Hybrid Feature Selection Scheme for Reducing Diagnostic Performance Deterioration Caused by Outliers in Data-Driven Diagnostics","volume":"63","author":"Kang","year":"2016","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"7067","DOI":"10.1109\/TIE.2016.2582729","article-title":"Real-Time Motor Fault Detection by 1-D Convolutional Neural Networks","volume":"63","author":"Ince","year":"2016","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"110030","DOI":"10.1016\/j.measurement.2021.110030","article-title":"A novel feature adaptive extraction method based on deep learning for bearing fault diagnosis","volume":"185","author":"Zhang","year":"2021","journal-title":"Measurement"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"141","DOI":"10.3389\/fpls.2020.00141","article-title":"Doing More With Less: A Multitask Deep Learning Approach in Plant Phenotyping","volume":"11","author":"Dobrescu","year":"2020","journal-title":"Front. Plant Sci."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1345","DOI":"10.1109\/TKDE.2009.191","article-title":"A survey on transfer learning","volume":"22","author":"Pan","year":"2009","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"485","DOI":"10.1016\/j.ymssp.2010.07.017","article-title":"Rolling element bearing diagnostics-A tutorial","volume":"25","author":"Randall","year":"2011","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Pang, B., Tang, G., Tian, T., and Zhou, C. (2018). Rolling Bearing Fault Diagnosis Based on an Improved HTT Transform. Sensors, 18.","DOI":"10.3390\/s18041203"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"4181","DOI":"10.1109\/JSEN.2019.2898634","article-title":"Physics-based convolutional neural network for fault diagnosis of rolling element bearings","volume":"19","author":"Sadoughi","year":"2019","journal-title":"IEEE Sens. J."},{"key":"ref_29","unstructured":"Howard, I. (1994). A Review of Rolling Element Bearing Vibration, Detection, Diagnosis and Prognosis, DATO. DSTO-RR-00113."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"Lecun","year":"2015","journal-title":"Nature"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"620","DOI":"10.1016\/j.measurement.2019.02.075","article-title":"Acoustic spectral imaging and transfer learning for reliable bearing fault diagnosis under variable speed conditions","volume":"138","author":"Hasan","year":"2019","journal-title":"Meas. J. Int. Meas. Confed."},{"key":"ref_32","first-page":"1929","article-title":"Dropout: A simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_33","unstructured":"Ioffe, S., and Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Dahl, G.E., Sainath, T.N., and Hinton, G.E. (2013, January 26\u201331). Improving deep neural networks for LVCSR using rectified linear units and dropout. Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada.","DOI":"10.1109\/ICASSP.2013.6639346"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"2377","DOI":"10.1109\/TIM.2019.2956332","article-title":"A New Intelligent Bearing Fault Diagnosis Method Using SDP Representation and SE-CNN","volume":"69","author":"Wang","year":"2019","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"4696","DOI":"10.1109\/TIE.2018.2866050","article-title":"Multiple Wavelet Coefficients Fusion in Deep Residual Networks for Fault Diagnosis","volume":"66","author":"Zhao","year":"2019","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"42373","DOI":"10.1109\/ACCESS.2019.2907131","article-title":"A deep learning method for bearing fault diagnosis based on time-frequency image","volume":"7","author":"Wang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.measurement.2017.07.017","article-title":"A convolutional neural network based feature learning and fault diagnosis method for the condition monitoring of gearbox","volume":"111","author":"Jing","year":"2017","journal-title":"Measurement"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1016\/j.ultras.2016.09.011","article-title":"A pre-trained convolutional neural network based method for thyroid nodule diagnosis","volume":"73","author":"Ma","year":"2017","journal-title":"Ultrasonics"},{"key":"ref_40","unstructured":"Brownlee, J. (2018). What is the Difference Between a Batch and an Epoch in a Neural Network?. Mach. Learn. Mastery, Available online: https:\/\/machinelearningmastery.com\/difference-between-a-batch-and-an-epoch\/."},{"key":"ref_41","unstructured":"Ruder, S. (2017). An overview of multi-task learning in deep neural networks. arXiv."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Hasan, M.J., and Kim, J.-M. (2018). Bearing Fault Diagnosis under Variable Rotational Speeds Using Stockwell Transform-Based Vibration Imaging and Transfer Learning. Appl. Sci., 8.","DOI":"10.3390\/app8122357"},{"key":"ref_43","unstructured":"Long, M., Cao, Z., Wang, J., and Philip, S.Y. (2017, January 4). Learning multiple tasks with multilinear relationship networks. Proceedings of the Advances in Neural Information Processing Systems, Online."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1016\/j.cogsys.2018.03.002","article-title":"Rolling element bearing fault diagnosis using convolutional neural network and vibration image","volume":"53","author":"Hoang","year":"2019","journal-title":"Cogn. Syst. Res."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Hasan, M.J., and Kim, J.-M. (2019, January 18). Deep Convolutional Neural Network with 2D Spectral Energy Maps for Fault Diagnosis of Gearboxes under Variable Speed. Proceedings of the Mediterranean Conference on Pattern Recognition and Artificial Intelligence, Online.","DOI":"10.1007\/978-3-030-37548-5_9"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"7286","DOI":"10.1016\/j.jfranklin.2020.04.024","article-title":"An unsupervised fault diagnosis method for rolling bearing using STFT and generative neural networks","volume":"357","author":"Tao","year":"2020","journal-title":"J. Frankl. Inst."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1109\/JSEE.2013.00004","article-title":"Selection of optimal window length using STFT for quantitative SNR analysis of LFM signal","volume":"24","author":"Yin","year":"2013","journal-title":"J. Syst. Eng. Electron."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"26241","DOI":"10.1109\/ACCESS.2018.2837621","article-title":"Preprocessing-Free Gear Fault Diagnosis Using Small Datasets with Deep Convolutional Neural Network-Based Transfer Learning","volume":"6","author":"Cao","year":"2018","journal-title":"IEEE Access"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1016\/j.patcog.2019.02.023","article-title":"The impact of class imbalance in classification performance metrics based on the binary confusion matrix","volume":"91","author":"Luque","year":"2019","journal-title":"Pattern Recognit."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Goutte, C., and Gaussier, E. (2005, January 21\u201323). A probabilistic interpretation of precision, recall and F-score, with implication for evaluation. Proceedings of the European Conference on Information Retrieval, Santiago de Compostela, Spain.","DOI":"10.1007\/978-3-540-31865-1_25"},{"key":"ref_51","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"Hinton","year":"2008","journal-title":"J. Mach. Learn. Res."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1006\/jmps.1999.1279","article-title":"Cross-validation methods","volume":"44","author":"Browne","year":"2000","journal-title":"J. Math. Psychol."},{"key":"ref_53","unstructured":"Case Western Reserve University (2021, November 13). Case Western Bearing Data Center. Available online: https:\/\/engineering.case.edu\/bearingdatacenter."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"3334","DOI":"10.1109\/TIM.2019.2933342","article-title":"Fault diagnosis of rotary machine bearings under inconsistent working conditions","volume":"69","author":"Sohaib","year":"2019","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Hasan, M.J., Sohaib, M., and Kim, J.-M. (2021). An Explainable AI-Based Fault Diagnosis Model for Bearings. Sensors, 21.","DOI":"10.3390\/s21124070"},{"key":"ref_56","unstructured":"Piezotronic, P. (2021, November 13). Sensor Details. Available online: http:\/\/www.pcb.com\/contentstore\/mktgContent\/IMI_Downloads\/IM%0AI-RouteBased_LowRes.pdf."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"494","DOI":"10.1109\/TIE.2014.2327555","article-title":"Vibration spectrum imaging: A novel bearing fault classification approach","volume":"62","author":"Amar","year":"2015","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"3325","DOI":"10.1109\/TIM.2019.2933119","article-title":"A Motor Current Signal-Based Bearing Fault Diagnosis Using Deep Learning and Information Fusion","volume":"69","author":"Hoang","year":"2019","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"8005","DOI":"10.1109\/TIE.2019.2942548","article-title":"Multitask Convolutional Neural Network with Information Fusion for Bearing Fault Diagnosis and Localization","volume":"67","author":"Guo","year":"2019","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_60","first-page":"300","article-title":"Wavelet analysis in volcanology: The case of phlegrean fields","volume":"6","author":"Pucciarelli","year":"2017","journal-title":"J. Environ. Sci. Eng. A"},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"120297","DOI":"10.1109\/ACCESS.2021.3108972","article-title":"Rolling element fault diagnosis based on VMD and sensitivity MCKD","volume":"9","author":"Cui","year":"2021","journal-title":"IEEE Access"},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/j.inffus.2018.04.003","article-title":"Multi-sensor data fusion based on the belief divergence measure of evidences and the belief entropy","volume":"46","author":"Xiao","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1016\/j.inffus.2021.03.008","article-title":"A novel approach of multisensory fusion to collaborative fault diagnosis in maintenance","volume":"74","author":"Shao","year":"2021","journal-title":"Inf. Fusion"},{"key":"ref_64","first-page":"11294","article-title":"Tri-Partition State Alphabet-Based Sequential Pattern for Multivariate Time Series","volume":"11","author":"Zhang","year":"2021","journal-title":"Cognit. Comput."},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Ran, X., Zhou, X., Lei, M., Tepsan, W., and Deng, W. (2021). A novel k-means clustering algorithm with a noise algorithm for capturing urban hotspots. Appl. Sci., 11.","DOI":"10.3390\/app112311202"},{"key":"ref_66","unstructured":"Case Western Reserve University (2021, November 13). Bearing Data Center Website. Available online: http:\/\/csegroups.case.edu\/bearingdatacenter\/pages\/download-data-file."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/1\/56\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:51:26Z","timestamp":1760169086000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/1\/56"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,22]]},"references-count":66,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,1]]}},"alternative-id":["s22010056"],"URL":"https:\/\/doi.org\/10.3390\/s22010056","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12,22]]}}}