{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T21:35:02Z","timestamp":1778621702738,"version":"3.51.4"},"reference-count":41,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2024,4,24]],"date-time":"2024-04-24T00:00:00Z","timestamp":1713916800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Natural Science Foundation of Hubei Province of China","award":["2022CFB935"],"award-info":[{"award-number":["2022CFB935"]}]},{"name":"Natural Science Foundation of Hubei Province of China","award":["2022KJX10"],"award-info":[{"award-number":["2022KJX10"]}]},{"name":"Open Fund of Hubei Key Laboratory for Operation and Control of Cascaded Hydropower Station","award":["2022CFB935"],"award-info":[{"award-number":["2022CFB935"]}]},{"name":"Open Fund of Hubei Key Laboratory for Operation and Control of Cascaded Hydropower Station","award":["2022KJX10"],"award-info":[{"award-number":["2022KJX10"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>To address the problems of existing 2D image-based imbalanced fault diagnosis methods for rolling bearings, which generate images with inadequate texture details and color degradation, this paper proposes a novel image enhancement model based on a dual-branch generative adversarial network (GAN) combining spatial and frequency domain information for an imbalanced fault diagnosis of rolling bearing. Firstly, the original vibration signals are converted into 2D time\u2013frequency (TF) images by a continuous wavelet transform, and a dual-branch GAN model with a symmetric structure is constructed. One branch utilizes an auxiliary classification GAN (ACGAN) to process the spatial information of the TF images, while the other employs a GAN with a frequency generator and a frequency discriminator to handle the frequency information of the input images after a fast Fourier transform. Then, a shuffle attention (SA) module based on an attention mechanism is integrated into the proposed model to improve the network\u2019s expression ability and reduce the computational burden. Simultaneously, mean square error (MSE) is integrated into the loss functions of both generators to enhance the consistency of frequency information for the generated images. Additionally, a Wasserstein distance and gradient penalty are also incorporated into the losses of the two discriminators to prevent gradient vanishing and mode collapse. Under the supervision of the frequency WGAN-GP branch, an ACWGAN-GP can generate high-quality fault samples to balance the dataset. Finally, the balanced dataset is utilized to train the auxiliary classifier to achieve fault diagnosis. The effectiveness of the proposed method is validated by two rolling bearing datasets. When the imbalanced ratios of the four datasets are 0.5, 0.2, 0.1, and 0.05, respectively, their average classification accuracy reaches 99.35% on the CWRU bearing dataset. Meanwhile, the average classification accuracy reaches 96.62% on the MFS bearing dataset.<\/jats:p>","DOI":"10.3390\/sym16050512","type":"journal-article","created":{"date-parts":[[2024,4,24]],"date-time":"2024-04-24T03:56:31Z","timestamp":1713930991000},"page":"512","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Image Enhancement Based on Dual-Branch Generative Adversarial Network Combining Spatial and Frequency Domain Information for Imbalanced Fault Diagnosis of Rolling Bearing"],"prefix":"10.3390","volume":"16","author":[{"given":"Yuguang","family":"Huang","sequence":"first","affiliation":[{"name":"College of Electrical Engineering and New Energy, China Three Gorges University, Yichang 443002, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bin","family":"Wen","sequence":"additional","affiliation":[{"name":"College of Electrical Engineering and New Energy, China Three Gorges University, Yichang 443002, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-7381-3579","authenticated-orcid":false,"given":"Weiqing","family":"Liao","sequence":"additional","affiliation":[{"name":"College of Electrical Engineering and New Energy, China Three Gorges University, Yichang 443002, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yahui","family":"Shan","sequence":"additional","affiliation":[{"name":"Wuhan Second Ship Design and Research Institute, Wuhan 430064, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4084-1085","authenticated-orcid":false,"given":"Wenlong","family":"Fu","sequence":"additional","affiliation":[{"name":"College of Electrical Engineering and New Energy, China Three Gorges University, Yichang 443002, China"},{"name":"Hubei Provincial Key Laboratory for Operation and Control of Cascaded Hydropower Station, China Three Gorges University, Yichang 443002, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Renming","family":"Wang","sequence":"additional","affiliation":[{"name":"College of Electrical Engineering and New Energy, China Three Gorges University, Yichang 443002, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,4,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"108964","DOI":"10.1016\/j.ymssp.2022.108964","article-title":"Unknown fault feature extraction of rolling bearings under variable speed conditions based on statistical complexity measures","volume":"172","author":"Wang","year":"2022","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"045005","DOI":"10.1088\/1361-6501\/acabdb","article-title":"Rolling bearing fault diagnosis based on 2D time-frequency images and data augmentation technique","volume":"34","author":"Fu","year":"2023","journal-title":"Meas. Sci. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"104474","DOI":"10.1016\/j.engfailanal.2020.104474","article-title":"Failure analysis of tapered roller bearing inner rings used in heavy truck","volume":"111","author":"Xu","year":"2020","journal-title":"Eng. Fail. Anal."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"106545","DOI":"10.1016\/j.ymssp.2019.106545","article-title":"Fault feature extraction of rotating machinery using a reweighted complete ensemble empirical mode decomposition with adaptive noise and demodulation analysis","volume":"138","author":"Wang","year":"2020","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"119310","DOI":"10.1016\/j.renene.2023.119310","article-title":"Refined composite hierarchical multiscale Lempel-Ziv complexity: A quantitative diagnostic method of multi-feature fusion for rotating energy devices","volume":"218","author":"Zhao","year":"2023","journal-title":"Renew. Energy"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"106587","DOI":"10.1016\/j.ymssp.2019.106587","article-title":"Applications of machine learning to machine fault diagnosis: A review and roadmap","volume":"138","author":"Lei","year":"2020","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1109\/TIM.2017.2759418","article-title":"Intelligent bearing fault diagnosis method combining compressed data acquisition and deep learning","volume":"67","author":"Sun","year":"2017","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"056114","DOI":"10.1088\/1361-6501\/ad282a","article-title":"Multi-scale residual neural network with enhanced gated recurrent unit for fault diagnosis of rolling bearing","volume":"35","author":"Liao","year":"2024","journal-title":"Meas. Sci. Technol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"106752","DOI":"10.1016\/j.ymssp.2020.106752","article-title":"An enhanced selective ensemble deep learning method for rolling bearing fault diagnosis with beetle antennae search algorithm","volume":"142","author":"Li","year":"2020","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"186","DOI":"10.1016\/j.jmsy.2021.11.016","article-title":"Unsupervised domain-share CNN for machine fault transfer diagnosis from steady speeds to time-varying speeds","volume":"62","author":"Cao","year":"2022","journal-title":"J. Manuf. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"66257","DOI":"10.1109\/ACCESS.2020.2985617","article-title":"Deep convolutional and LSTM recurrent neural networks for rolling bearing fault diagnosis under strong noises and variable loads","volume":"8","author":"Qiao","year":"2020","journal-title":"IEEE Access"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"109569","DOI":"10.1016\/j.ymssp.2022.109569","article-title":"Discriminative feature learning using a multiscale convolutional capsule network from attitude data for fault diagnosis of industrial robots","volume":"182","author":"Long","year":"2023","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"110587","DOI":"10.1016\/j.measurement.2021.110587","article-title":"New transfer learning fault diagnosis method of rolling bearing based on ADC-CNN and LATL under variable conditions","volume":"188","author":"Huo","year":"2022","journal-title":"Measurement"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"110332","DOI":"10.1016\/j.measurement.2021.110332","article-title":"Joint distribution adaptation with diverse feature aggregation: A new transfer learning framework for bearing diagnosis across different machines","volume":"187","author":"Jia","year":"2022","journal-title":"Measurement"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zheng, T., Song, L., Guo, B., Liang, H., and Guo, L. (2019, January 25\u201327). An efficient method based on conditional generative adversarial networks for imbalanced fault diagnosis of rolling bearing. Proceedings of the 2019 Prognostics and System Health Management Conference (PHM-Qingdao), Qingdao, China.","DOI":"10.1109\/PHM-Qingdao46334.2019.8942906"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"9515","DOI":"10.1109\/ACCESS.2018.2890693","article-title":"Imbalanced fault diagnosis of rolling bearing based on generative adversarial network: A comparative study","volume":"7","author":"Mao","year":"2019","journal-title":"IEEE Access"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1195","DOI":"10.1109\/TR.2019.2942049","article-title":"A local adaptive minority selection and oversampling method for class-imbalanced fault diagnostics in industrial systems","volume":"69","author":"Wu","year":"2019","journal-title":"IEEE Trans. Reliab."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1016\/j.ymssp.2016.06.024","article-title":"Online sequential prediction of bearings imbalanced fault diagnosis by extreme learning machine","volume":"83","author":"Mao","year":"2017","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"6248","DOI":"10.1109\/TIE.2020.2994868","article-title":"Fault diagnosis of an autonomous vehicle with an improved SVM algorithm subject to unbalanced datasets","volume":"68","author":"Shi","year":"2020","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"107377","DOI":"10.1016\/j.measurement.2019.107377","article-title":"Machinery fault diagnosis with imbalanced data using deep generative adversarial networks","volume":"152","author":"Zhang","year":"2020","journal-title":"Measurement"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"487","DOI":"10.1016\/j.neucom.2018.10.109","article-title":"Data augmentation in fault diagnosis based on the Wasserstein generative adversarial network with gradient penalty","volume":"396","author":"Gao","year":"2020","journal-title":"Neurocomputing"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"062002","DOI":"10.1088\/1361-6501\/ad356d","article-title":"A Review: The Application of Generative Adversarial Network for Mechanical Fault Diagnosis","volume":"35","author":"Liao","year":"2024","journal-title":"Meas. Sci. Technol."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Liu, S., Dou, L., and Jin, Q. (2023, January 23\u201325). Improved Generative Adversarial Network for Bearing Fault Diagnosis with Imbalanced Data. Proceedings of the 2023 6th International Conference on Information Communication and Signal Processing (ICICSP), Xi\u2019an, China.","DOI":"10.1109\/ICICSP59554.2023.10390723"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Fu, W., Yang, K., Wen, B., Shan, Y., Li, S., and Zheng, B. (2024). Rotating Machinery Fault Diagnosis with Limited Multisensor Fusion Samples by Fused Attention-Guided Wasserstein GAN. Symmetry, 16.","DOI":"10.3390\/sym16030285"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"108664","DOI":"10.1016\/j.ymssp.2021.108664","article-title":"Imbalanced fault diagnosis of rolling bearing using improved MsR-GAN and feature enhancement-driven CapsNet","volume":"168","author":"Liu","year":"2022","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"27736","DOI":"10.1109\/ACCESS.2021.3058334","article-title":"An imbalanced fault diagnosis method for rolling bearing based on semi-supervised conditional generative adversarial network with spectral normalization","volume":"9","author":"Xu","year":"2021","journal-title":"IEEE Access"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"107768","DOI":"10.1016\/j.measurement.2020.107768","article-title":"Intelligent fault diagnosis of rotating machinery via wavelet transform, generative adversarial nets and convolutional neural network","volume":"159","author":"Liang","year":"2020","journal-title":"Measurement"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"19543","DOI":"10.1109\/JSEN.2022.3200691","article-title":"An intelligent fault diagnosis method of small sample bearing based on improved auxiliary classification generative adversarial network","volume":"22","author":"Meng","year":"2022","journal-title":"IEEE Sens. J."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"10706","DOI":"10.1109\/JSEN.2023.3263924","article-title":"Ensemble multiple distinct ResNet networks with channel-attention mechanism for multi-sensor fault diagnosis of hydraulic systems","volume":"23","author":"Peng","year":"2023","journal-title":"IEEE Sens. J."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Hou, C., Sun, Q., Wang, W., and Zhang, J. (2022, January 16\u201319). Shuffle Attention Multiple Instances Learning for Breast Cancer Whole Slide Image Classification. Proceedings of the 2022 IEEE International Conference on Image Processing (ICIP), Bordeaux, France.","DOI":"10.1109\/ICIP46576.2022.9897835"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1145\/3422622","article-title":"Generative adversarial networks","volume":"63","author":"Goodfellow","year":"2020","journal-title":"Commun. ACM"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Wu, C., and Zeng, Z. (2021). A fault diagnosis method based on Auxiliary Classifier Generative Adversarial Network for rolling bearing. PLoS ONE, 16.","DOI":"10.1371\/journal.pone.0246905"},{"key":"ref_33","unstructured":"Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., and Courville, A.C. (2017). Improved training of wasserstein gans. Adv. Neural Inf. Process. Syst., 30."},{"key":"ref_34","first-page":"3500417","article-title":"A novel method for imbalanced fault diagnosis of rotating machinery based on generative adversarial networks","volume":"70","author":"Li","year":"2020","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.sigpro.2013.04.015","article-title":"Wavelets for fault diagnosis of rotary machines: A review with applications","volume":"96","author":"Yan","year":"2014","journal-title":"Signal Process."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Nam, J., and Lee, S. (2021, January 19\u201322). Frequency filtering for data augmentation in X-ray image classification. Proceedings of the 2021 IEEE International Conference on Image Processing (ICIP), Anchorage, AK, USA.","DOI":"10.1109\/ICIP42928.2021.9506587"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Pu, Y., Wang, W., and Xu, Q. (2012, January 23\u201326). Image change detection based on the minimum mean square error. Proceedings of the 2012 Fifth International Joint Conference on Computational Sciences and Optimization, Harbin, China.","DOI":"10.1109\/CSO.2012.88"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Liang, X., and Guo, L. (2022, January 14\u201316). Fault diagnosis of batch process based on improved time convolution network and efficient channel attention. Proceedings of the 2022 4th International Conference on Intelligent Information Processing (IIP), Guangzhou, China.","DOI":"10.1109\/IIP57348.2022.00033"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"9136","DOI":"10.1109\/JSEN.2023.3251654","article-title":"Intelligent Fault Diagnosis of Rolling Bearings Using Efficient and Lightweight ResNet Networks Based on an Attention Mechanism","volume":"23","author":"Chang","year":"2023","journal-title":"IEEE Sens. J."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1016\/j.jsv.2016.09.012","article-title":"Fault diagnosis of motor bearing with speed fluctuation via angular resampling of transient sound signals","volume":"385","author":"Lu","year":"2016","journal-title":"J. Sound Vib."},{"key":"ref_41","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"Maaten","year":"2008","journal-title":"J. Mach. Learn. Res."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/16\/5\/512\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:33:03Z","timestamp":1760106783000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/16\/5\/512"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,24]]},"references-count":41,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2024,5]]}},"alternative-id":["sym16050512"],"URL":"https:\/\/doi.org\/10.3390\/sym16050512","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,24]]}}}