{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T14:18:52Z","timestamp":1774448332933,"version":"3.50.1"},"reference-count":41,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2024,10,25]],"date-time":"2024-10-25T00:00:00Z","timestamp":1729814400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100017700","name":"Henan Provincial Science and Technology Research Project","doi-asserted-by":"publisher","award":["222102220107"],"award-info":[{"award-number":["222102220107"]}],"id":[{"id":"10.13039\/501100017700","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100017700","name":"Henan Provincial Science and Technology Research Project","doi-asserted-by":"publisher","award":["242102210101"],"award-info":[{"award-number":["242102210101"]}],"id":[{"id":"10.13039\/501100017700","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>This study presents a new image inpainting model based on U-Net and incorporating the Wasserstein Generative Adversarial Network (WGAN). The model uses skip connections to connect every encoder block to the corresponding decoder block, resulting in a strictly symmetrical architecture referred to as Symmetric Connected U-Net (SC-Unet). By combining SC-Unet with a GAN, the study aims to reconstruct images more effectively and seamlessly. The traditional discriminators only differentiate the entire image as true or false. In this study, the discriminator calculated the probability of each pixel belonging to the hole and non-hole regions, which provided the generator with more gradient loss information for image inpainting. Additionally, every block of SC-Unet incorporated a Dilated Convolutional Neural Network (DCNN) to increase the receptive field of the convolutional layers. Our model also integrated Multi-Head Self-Attention (MHSA) into selected blocks to enable it to efficiently search the entire image for suitable content to fill the missing areas. This study adopts the publicly available datasets CelebA-HQ and ImageNet for evaluation. Our proposed algorithm demonstrates a 10% improvement in PSNR and a 2.94% improvement in SSIM compared to existing representative image inpainting methods in the experiment.<\/jats:p>","DOI":"10.3390\/sym16111423","type":"journal-article","created":{"date-parts":[[2024,10,25]],"date-time":"2024-10-25T07:51:29Z","timestamp":1729842689000},"page":"1423","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Symmetric Connected U-Net with Multi-Head Self Attention (MHSA) and WGAN for Image Inpainting"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3072-5227","authenticated-orcid":false,"given":"Yanyang","family":"Hou","sequence":"first","affiliation":[{"name":"School of Information Engineering, Zhengzhou University of Industrial Technology, Zhengzhou 451100, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiaopeng","family":"Ma","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Zhengzhou University of Technology, Zhengzhou 450044, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Junjun","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Software Engineering, Henan University of Engineering, Zhengzhou 451191, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chenxian","family":"Guo","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Zhengzhou University of Technology, Zhengzhou 450044, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,10,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Sun, J., Yuan, L., and Jia, J. (2005). Image completion with structure propagation. ACM SIGGRAPH 2005 Papers, Association for Computing Machinery.","DOI":"10.1145\/1186822.1073274"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_3","unstructured":"Xu, Y., Gu, T., and Chen, W. (2024). Ootdiffusion: Outfitting fusion based latent diffusion for controllable virtual try-on. arXiv."},{"key":"ref_4","first-page":"102","article-title":"Advances in digital image inpainting algorithms based on deep learning","volume":"36","author":"Chunqi","year":"2020","journal-title":"J. Signal Process"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"102028","DOI":"10.1016\/j.displa.2021.102028","article-title":"Image inpainting based on deep learning: A review","volume":"69","author":"Qin","year":"2021","journal-title":"Displays"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"111343","DOI":"10.1016\/j.knosys.2023.111343","article-title":"Lightweight image super-resolution for IoT devices using deep residual feature distillation network","volume":"285","author":"Mardieva","year":"2024","journal-title":"Knowl.-Based Syst."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_8","unstructured":"Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative adversarial networks. arXiv."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 1\u201326). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1016\/j.neucom.2018.11.045","article-title":"Multi-scale semantic image inpainting with residual learning and GAN","volume":"331","author":"Jiao","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"e21","DOI":"10.23915\/distill.00021","article-title":"Computing receptive fields of convolutional neural networks","volume":"4","author":"Araujo","year":"2019","journal-title":"Distill"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1873","DOI":"10.1109\/LSP.2021.3109774","article-title":"Diverse receptive field based adversarial concurrent encoder network for image inpainting","volume":"28","author":"Phutke","year":"2021","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_13","unstructured":"Yu, F. (2015). Multi-scale context aggregation by dilated convolutions. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Yan, Z., Li, X., and Li, M. (2018, January 8\u201314). Shift-net: Image inpainting via deep feature rearrangement. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01264-9_1"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Yu, J., and Lin, Z. (2018, January 18\u201323). Generative image inpainting with contextual attention. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00577"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Liu, W., Shi, Y., and Li, J. (2023, January 4\u20136). Multi-stage Progressive Reasoning for Dunhuang Murals Inpainting. Proceedings of the 2023 IEEE 4th International Conference on Pattern Recognition and Machine Learning (PRML), Urumqi, China.","DOI":"10.1109\/PRML59573.2023.10348363"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Liu, G., Reda, F.A., and Shih, K.J. (2018, January 8\u201314). Image inpainting for irregular holes using partial convolutions. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01252-6_6"},{"key":"ref_18","unstructured":"Pathak, D., Krahenbuhl, P., and Donahue, J. (July, January 26). Context encoders: Feature learning by inpainting. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_19","unstructured":"Im, D.J., Kim, C.D., and Jiang, H. (2016). Generating Images with Recurrent Adversarial Networks. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3072959.3073659","article-title":"Globally and locally consistent image completion","volume":"36","author":"Iizuka","year":"2017","journal-title":"ACM Trans. Graph"},{"key":"ref_21","unstructured":"Radford, A., Metz, L., and Chintala, S. (2015). Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv."},{"key":"ref_22","unstructured":"Arjovsky, M., and Chintala, S. (2017, January 6\u201311). Wasserstein generative adversarial networks. Proceedings of the International Conference on Machine Learning, Sydney, NSW, Australia."},{"key":"ref_23","unstructured":"Mirza, M. (2014). Conditional generative adversarial nets. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Zhu, J.-Y., Park, T., and Isola, P. (2017, January 22\u201329). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.244"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Lou, S., Fan, Q., and Chen, F. (2018, January 19\u201320). Preliminary investigation on single remote sensing image inpainting through a modified GAN. Proceedings of the 2018 10th IAPR Workshop on Pattern Recognition in Remote Sensing, PRRS, Beijing, China.","DOI":"10.1109\/PRRS.2018.8486163"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Deng, Y., Hui, S., and Zhou, S. (2021, January 20\u201324). Learning contextual transformer network for image inpainting. Proceedings of the 29th ACM International Conference on Multimedia, Chengdu, China.","DOI":"10.1145\/3474085.3475426"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"107448","DOI":"10.1016\/j.patcog.2020.107448","article-title":"Multistage attention network for image inpainting","volume":"106","author":"Wang","year":"2020","journal-title":"Pattern Recognit."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Li, J., Wang, N., and Zhang, L. (2020, January 13\u201319). Recurrent feature reasoning for image inpainting. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00778"},{"key":"ref_29","unstructured":"Liu, H., and Jiang, B. (November, January 27). Coherent semantic attention for image inpainting. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Wang, W., Xie, E., and Li, X. (2021, January 10\u201317). Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.00061"},{"key":"ref_31","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Wang, Q., He, S., and Su, M. (2024). Context-Encoder-Based Image Inpainting for Ancient Chinese Silk. Appl. Sci., 14.","DOI":"10.3390\/app14156607"},{"key":"ref_33","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"Imagenet large scale visual recognition challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Gatys, L.A., Ecker, A.S., and Bethge, M. (2015). A neural algorithm of artistic style. arXiv.","DOI":"10.1167\/16.12.326"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Gatys, L.A., Ecker, A.S., and Bethge, M. (2016, January 27\u201330). Image style transfer using convolutional neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.265"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Johnson, J., Alahi, A., and Fei-Fei, L. (2016, January 11\u201314). Perceptual losses for real-time style transfer and super-resolution. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46475-6_43"},{"key":"ref_38","unstructured":"Karras, T., Aila, T., and Laine, S. (2017). Progressive growing of gans for improved quality, stability, and variation. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Hore, A., and Ziou, D. (2010, January 23\u201326). Image quality metrics: PSNR vs. SSIM. Proceedings of the 2010 20th International Conference on Pattern Recognition (ICPR), IEEE, Istanbul, Turkey.","DOI":"10.1109\/ICPR.2010.579"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_41","unstructured":"Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. (2017). Gans trained by a two time-scale update rule converge to a local nash equilibrium. arXiv."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/16\/11\/1423\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:20:24Z","timestamp":1760113224000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/16\/11\/1423"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,25]]},"references-count":41,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2024,11]]}},"alternative-id":["sym16111423"],"URL":"https:\/\/doi.org\/10.3390\/sym16111423","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,25]]}}}