{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T18:36:41Z","timestamp":1780684601192,"version":"3.54.1"},"reference-count":43,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2024,3,11]],"date-time":"2024-03-11T00:00:00Z","timestamp":1710115200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"the science and technology research project of Nanchang Institute of Science and Technology","award":["NGKJ-23-01"],"award-info":[{"award-number":["NGKJ-23-01"]}]},{"name":"the science and technology research project of Nanchang Institute of Science and Technology","award":["NGRCZX-23-09"],"award-info":[{"award-number":["NGRCZX-23-09"]}]},{"name":"the science and technology research project of Nanchang Institute of Science and Technology","award":["NGYJZX-2021-04"],"award-info":[{"award-number":["NGYJZX-2021-04"]}]},{"name":"the initial scientific research foundation for talented scholars of Nanchang Institute of Science and Technology","award":["NGKJ-23-01"],"award-info":[{"award-number":["NGKJ-23-01"]}]},{"name":"the initial scientific research foundation for talented scholars of Nanchang Institute of Science and Technology","award":["NGRCZX-23-09"],"award-info":[{"award-number":["NGRCZX-23-09"]}]},{"name":"the initial scientific research foundation for talented scholars of Nanchang Institute of Science and Technology","award":["NGYJZX-2021-04"],"award-info":[{"award-number":["NGYJZX-2021-04"]}]},{"name":"the nonlinear dynamics and application research center project of Nanchang Institute of Science and Technology","award":["NGKJ-23-01"],"award-info":[{"award-number":["NGKJ-23-01"]}]},{"name":"the nonlinear dynamics and application research center project of Nanchang Institute of Science and Technology","award":["NGRCZX-23-09"],"award-info":[{"award-number":["NGRCZX-23-09"]}]},{"name":"the nonlinear dynamics and application research center project of Nanchang Institute of Science and Technology","award":["NGYJZX-2021-04"],"award-info":[{"award-number":["NGYJZX-2021-04"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Deep learning (DL) has been widely used to promote the development of intelligent fault diagnosis, bringing significant performance improvement. However, most of the existing methods cannot capture the temporal information and global features of mechanical equipment to collect sufficient fault information, resulting in performance collapse. Meanwhile, due to the complex and harsh operating environment, it is difficult to extract fault features stably and extensively using single-source fault diagnosis methods. Therefore, a novel hierarchical vision transformer (NHVT) and wavelet time\u2013frequency architecture combined with a multi-source information fusion (MSIF) strategy has been suggested in this paper to boost stable performance by extracting and integrating rich features. The goal is to improve the end-to-end fault diagnosis performance of mechanical components. First, multi-source signals are transformed into two-dimensional time and frequency diagrams. Then, a novel hierarchical vision transformer is introduced to improve the nonlinear representation of feature maps to enrich fault features. Next, multi-source information diagrams are fused into the proposed NHVT to produce more comprehensive presentations. Finally, we employed two different multi-source datasets to verify the superiority of the proposed NHVT. Then, NHVT outperformed the state-of-the-art approach (SOTA) on the multi-source dataset of mechanical components, and the experimental results show that it is able to extract useful features from multi-source information.<\/jats:p>","DOI":"10.3390\/s24061799","type":"journal-article","created":{"date-parts":[[2024,3,11]],"date-time":"2024-03-11T08:56:41Z","timestamp":1710147401000},"page":"1799","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["A Novel Hierarchical Vision Transformer and Wavelet Time\u2013Frequency Based on Multi-Source Information Fusion for Intelligent Fault Diagnosis"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9878-4856","authenticated-orcid":false,"given":"Changfen","family":"Gong","sequence":"first","affiliation":[{"name":"School of Education, Nanchang Institute of Science and Technology, Nanchang 330108, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7144-896X","authenticated-orcid":false,"given":"Rongrong","family":"Peng","sequence":"additional","affiliation":[{"name":"School of Education, Nanchang Institute of Science and Technology, Nanchang 330108, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,3,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"5914","DOI":"10.1109\/TMECH.2022.3191051","article-title":"Deep negative correlation multisource domains adaptation network for machinery fault diagnosis under different working conditions","volume":"27","author":"Ye","year":"2022","journal-title":"IEEE-ASME. Trans. Mech."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"111194","DOI":"10.1016\/j.ymssp.2024.111194","article-title":"A new multi-source information domain adaption network based on domain attributes and features transfer for cross-domain fault diagnosis","volume":"211","author":"Yu","year":"2024","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"116079","DOI":"10.1016\/j.jsv.2021.116079","article-title":"Novel self-adaptive vibration signal analysis: Concealed component decomposition and its application in bearing fault diagnosis","volume":"502","author":"Tiwari","year":"2021","journal-title":"J. Sound. Vib."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Peng, R.R., Zhang, X.Z., and Shi, P.M. (2023). Multi-representation domain adaptation network with duplex adversarial learning for hot-rolling mill fault diagnosis. Entropy, 25.","DOI":"10.3390\/e25010083"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Peng, R.R., Zhang, X.Z., and Shi, P.M. (2022). Bearing fault diagnosis of hot-rolling mill utilizing intelligent optimized self-adaptive deep belief network with limited samples. Sensors, 22.","DOI":"10.3390\/s22207815"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"8767","DOI":"10.1109\/TPEL.2022.3153797","article-title":"A machine-learning-based fault diagnosis method with adaptive secondary sampling for multiphase drive systems","volume":"37","author":"Liu","year":"2022","journal-title":"IEEE. Trans. Power Electron."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"108525","DOI":"10.1016\/j.ress.2022.108525","article-title":"Towards trustworthy machine fault diagnosis: A probabilistic Bayesian deep learning framework","volume":"224","author":"Zhou","year":"2022","journal-title":"Reliab. Eng. Syst. Saf."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1727","DOI":"10.1109\/JSEN.2020.3015868","article-title":"Infrared thermography-based fault diagnosis of induction motor bearings using machine learning","volume":"21","author":"Choudhary","year":"2021","journal-title":"IEEE. Sens. J."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"186","DOI":"10.1016\/j.jmsy.2021.11.016","article-title":"Unsupervised domain-share CNN for machine fault transfer diagnosis from steady speeds to time-varying speeds","volume":"62","author":"Cao","year":"2022","journal-title":"J. Manuf. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"101554","DOI":"10.1016\/j.aei.2022.101554","article-title":"A novel adaptive convolutional neural network for fault diagnosis of hydraulic piston pump with acoustic images","volume":"52","author":"Tang","year":"2022","journal-title":"Adv. Eng. Inform."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"485","DOI":"10.1016\/j.isatra.2021.11.024","article-title":"Rolling bearing fault diagnosis based on SSA optimized self-adaptive DBN","volume":"128","author":"Gao","year":"2022","journal-title":"ISA Trans."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"457","DOI":"10.1016\/j.isatra.2019.07.001","article-title":"A novel deep learning-based fault diagnosis approach for chemical process with extended deep belief network","volume":"96","author":"Wang","year":"2020","journal-title":"ISA Trans."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"869","DOI":"10.1109\/TSMC.2020.3005433","article-title":"An improved sparse autoencoder and multilevel denoising strategy for diagnosing early multiple intermittent faults","volume":"52","author":"Xie","year":"2022","journal-title":"IEEE. Trans. Syst. Man Cybern-Syst."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"4167","DOI":"10.1109\/TIE.2021.3075871","article-title":"SuperGraph: Spatial-temporal graph-based feature extraction for rotating machinery diagnosis","volume":"69","author":"Yang","year":"2022","journal-title":"IEEE. Trans. Ind. Electron."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2507","DOI":"10.1007\/s42835-022-01032-3","article-title":"Creating knowledge graph of electric power equipment faults based on BERT-BiLSTM-CRF model","volume":"17","author":"Meng","year":"2022","journal-title":"J. Electron. Eng. Technol."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Wang, X.L., Girshick, R., Gupta, A., and He, K. (2018). Non-local neural networks. arXiv.","DOI":"10.1109\/CVPR.2018.00813"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"124002","DOI":"10.1088\/1361-6501\/ac1b43","article-title":"Novel multi-scale dilated CNN-LSTM for fault diagnosis of planetary gearbox with unbalanced samples under noisy environment","volume":"32","author":"Han","year":"2021","journal-title":"Meas. Sci. Technol."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"9557","DOI":"10.1109\/TIM.2020.3005113","article-title":"Detecting the early damages in structures with nonlinear output frequency response functions and the CNN-LSTM model","volume":"69","author":"Zhao","year":"2020","journal-title":"IEEE. Trans. Instrum. Meas."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"100378","DOI":"10.1016\/j.compind.2020.103378","article-title":"Fault detection and diagnosis for rotating machinery: A model based on convolutional LSTM, fast fourier and continuous wavelet transforms","volume":"125","author":"Jalayer","year":"2021","journal-title":"Comput. Ind."},{"key":"ref_20","first-page":"2476","article-title":"Bridging text and video: A universal multimodal transformer for audio-visual scene-aware dialog","volume":"29","author":"Li","year":"2021","journal-title":"IEEE-ACM Trans. Audio Speech"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"108616","DOI":"10.1016\/j.ymssp.2021.108616","article-title":"A novel time-frequency transformer based on self-attention mechanism and its application in fault diagnosis of rolling bearings","volume":"168","author":"Ding","year":"2021","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"340","DOI":"10.1109\/TMECH.2022.3199985","article-title":"Convformer-NSE: A novel end-to-end gearbox fault diagnosis framework under heavy noise using joint global and local information","volume":"28","author":"Han","year":"2022","journal-title":"IEEE-ASME Trans. Mech."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1105445","DOI":"10.1016\/j.measurement.2021.110545","article-title":"Fault diagnosis based on SPBO-SDAE and transformer neural network for rotating machinery","volume":"188","author":"Du","year":"2022","journal-title":"Measurement"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Arellano-Espitia, F., Delgado-Prieto, M., Martinez-Viol, V., Savcedo-Dorantes, J.J., and Osornio-Rios, R.A. (2020). Deep-learning-based methodology for fault diagnosis in electromechanical systems. Sensors, 20.","DOI":"10.3390\/s20143949"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"110759","DOI":"10.1016\/j.measurement.2022.110759","article-title":"Fault detection and diagnosis in electric motors using 1d convolutional neural networks with multi-channel vibration signals","volume":"190","author":"Ribeiro","year":"2022","journal-title":"Measurement"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"110720","DOI":"10.1016\/j.measurement.2022.110720","article-title":"An improved multi-channel graph convolutional network and its applications for rotating machinery diagnosis","volume":"190","author":"Yang","year":"2022","journal-title":"Measurement"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"075901","DOI":"10.1088\/1361-6501\/ac5deb","article-title":"Bearing fault diagnosis method based on multi-source heterogeneous information fusion","volume":"33","author":"Zhang","year":"2022","journal-title":"Meas. Sci. Technol."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"055115","DOI":"10.1088\/1361-6501\/ac48eb","article-title":"A novel multi-information decision fusion based on improved random forests in HVCB fault detection application","volume":"33","author":"Ma","year":"2022","journal-title":"Meas. Sci. Technol."},{"key":"ref_29","first-page":"1094","article-title":"Planetary gearbox fault diagnosis technique based on multi-source information deep fusion","volume":"33","author":"Chen","year":"2020","journal-title":"J. Vib. Eng."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"3213","DOI":"10.1109\/TII.2021.3102017","article-title":"Intelligent mechanical fault diagnosis using multi-sensor fusion and convolution neural network","volume":"18","author":"Xie","year":"2021","journal-title":"IEEE Trans. Ind. Inf."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"116355","DOI":"10.1016\/j.trac.2021.116355","article-title":"Data handling in data fusion: Methodologies and applications","volume":"143","author":"Azcarate","year":"2021","journal-title":"TRAC-Trends Anal. Chem."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2446","DOI":"10.1109\/TII.2018.2864759","article-title":"Highly accurate machine fault diagnosis using deep transfer learning","volume":"15","author":"Shao","year":"2019","journal-title":"IEEE. Trans. Ind. Informa."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1016\/j.ymssp.2013.01.017","article-title":"Recent advances in time-frequency analysis methods for machinery fault diagnosis: A review with application examples","volume":"38","author":"Feng","year":"2013","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.apenergy.2013.09.043","article-title":"Multi-source information fusion based fault diagnosis of ground-source heat pump using Bayesian network","volume":"114","author":"Cai","year":"2014","journal-title":"Appl. Energy"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1016\/j.measurement.2017.02.047","article-title":"Quaternion singular spectrum analysis using convex optimization and its application to fault diagnosis of rolling bearing","volume":"103","author":"Yi","year":"2017","journal-title":"Measurement"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"16591","DOI":"10.1007\/s11042-022-13820-0","article-title":"The effect of choosing optimizer algorithms to improve computer vision tasks: A comparative study","volume":"82","author":"Hassan","year":"2023","journal-title":"Multimed. Tools. Appl."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Liu, Z., Lin, Y.T., Cao, Y., Hu, H., Wei, Y.X., Zhang, Z., Stephen, L., and Baining, G. (2021). SwinTransformer: Hierarchical vision transformer using shifted windows. arXiv.","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Fu, Y., Chen, X., Liu, Y., Son, C., and Yan, Y. (2022). Multi-source information fusion fault diagnosis for gearboxes based on SDP and VGG. Appl. Sci., 12.","DOI":"10.3390\/app12136323"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"108903","DOI":"10.1016\/j.ymssp.2022.108903","article-title":"A novel multi-source sensing data fusion driven method for detecting rolling mill health states under imbalanced and limited datasets","volume":"171","author":"Shi","year":"2022","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1016\/j.isatra.2022.08.002","article-title":"Rolling mill health states diagnosing method based on multi-sensor information fusion and improved DBNs under limited datasets","volume":"134","author":"Yu","year":"2023","journal-title":"ISA. Trans."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"6374","DOI":"10.2514\/1.J061219","article-title":"Sound propagation of three-dimensional sandwich panels: Influence of three-dimensional re-entrant auxetic core","volume":"60","author":"Ghafouri","year":"2022","journal-title":"AIAA J."},{"key":"ref_42","unstructured":"Keskar, N., and Socher, R. (2017). Improving generalization performance by switching from Adam to SGD. arXiv."},{"key":"ref_43","unstructured":"Wilson, A., Roelofs, R., and Stern, M. (2017, January 4\u20139). The marginal value of adaptive gradient methods in machine learning. Proceedings of the Advances in Neural Information Processing Systems 30, NIPS 2017, Long Beach, CA, USA."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/6\/1799\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:11:59Z","timestamp":1760105519000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/6\/1799"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,11]]},"references-count":43,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2024,3]]}},"alternative-id":["s24061799"],"URL":"https:\/\/doi.org\/10.3390\/s24061799","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,11]]}}}