{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T02:23:48Z","timestamp":1778811828506,"version":"3.51.4"},"reference-count":33,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2022,2,1]],"date-time":"2022-02-01T00:00:00Z","timestamp":1643673600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61972206,62011540407"],"award-info":[{"award-number":["61972206,62011540407"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Existing image inpainting methods based on deep learning have made great progress. These methods either generate contextually semantically consistent images or visually excellent images, ignoring that both semantic and visual effects should be appreciated. In this article, we propose a Semantic Residual Pyramid Network (SRPNet) based on a deep generative model for image inpainting at the image and feature levels. This method encodes a masked image by a residual semantic pyramid encoder and then decodes the encoded features into a inpainted image by a multi-layer decoder. At this stage, a multi-layer attention transfer network is used to gradually fill in the missing regions of the image. To generate semantically consistent and visually superior images, the multi-scale discriminators are added to the network structure. The discriminators are divided into global and local discriminators, where the global discriminator is used to identify the global consistency of the inpainted image, and the local discriminator is used to determine the consistency of the missing regions of the inpainted image. Finally, we conducted experiments on four different datasets. As a result, great performance was achieved for filling both the regular and irregular missing regions.<\/jats:p>","DOI":"10.3390\/info13020071","type":"journal-article","created":{"date-parts":[[2022,2,1]],"date-time":"2022-02-01T09:59:21Z","timestamp":1643709561000},"page":"71","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Semantic Residual Pyramid Network for Image Inpainting"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2033-9223","authenticated-orcid":false,"given":"Haiyin","family":"Luo","sequence":"first","affiliation":[{"name":"School of Computer Science, Nanjing University of Information Science & Technology, Nanjing 210044, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4408-3800","authenticated-orcid":false,"given":"Yuhui","family":"Zheng","sequence":"additional","affiliation":[{"name":"Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science & Technology, Nanjing 210044, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,2,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Bertalmio, M., Sapiro, G., Caselles, V., and Ballester, C. (2000, January 23\u201328). Image inpainting. Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, New Orleans, LA, USA.","DOI":"10.1145\/344779.344972"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1016\/j.image.2015.08.008","article-title":"A bibliography of pixel-based blind image forgery detection techniques","volume":"39","author":"Qureshi","year":"2015","journal-title":"Signal Process. Image Commun."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1016\/j.jvcir.2017.09.006","article-title":"A critical survey of state-of-the-art image inpainting quality assessment metrics","volume":"49","author":"Qureshi","year":"2017","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1019","DOI":"10.1137\/S0036139900368844","article-title":"Mathematical Models for Local Nontexture Inpaintings","volume":"62","author":"Shen","year":"2002","journal-title":"SIAM J. Appl. Math."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1006\/jvci.2001.0487","article-title":"Nontexture Inpainting by Curvature-Driven Diffusions","volume":"12","author":"Chan","year":"2001","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_6","unstructured":"Efros, A.A., and Freeman, W.T. Image quilting for texture synthesis and transfer. Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, New York, NY, USA."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1145\/1531326.1531330","article-title":"PatchMatch: A randomized correspondence algorithm for structural image editing","volume":"28","author":"Barnes","year":"2009","journal-title":"ACM Trans. Graph."},{"key":"ref_8","first-page":"769","article-title":"Natural image denoising with convolutional networks","volume":"21","author":"Jain","year":"2008","journal-title":"Adv. Neural Inf. Processing Syst."},{"key":"ref_9","unstructured":"Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., and Efros, A.A. (July, January 27). Context encoders: Feature learning by inpainting. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3072959.3073659","article-title":"Globally and locally consistent image completion","volume":"36","author":"Iizuka","year":"2017","journal-title":"ACM Trans. Graph."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Yi, Z., Tang, Q., Azizi, S., Jang, D., and Xu, Z. (2020, January 14\u201319). Contextual residual aggregation for ultra high-resolution image inpainting. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00753"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zeng, Y., Fu, J., Chao, H., and Guo, B. (2019, January 15\u201320). Learning pyramid-context encoder network for high-quality image inpainting. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00158"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"022022","DOI":"10.1088\/1742-6596\/1168\/2\/022022","article-title":"An Overview of Overfitting and its Solutions","volume":"1168","author":"Ying","year":"2019","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_15","unstructured":"Nazeri, K., Ng, E., Joseph, T., Qureshi, F.Z., and Ebrahimi, M. (2019). Edgeconnect: Generative image inpainting with adversarial edge learning. arXiv."},{"key":"ref_16","unstructured":"Ulyanov, D., Vedaldi, A., and Lempitsky, V. (2016). Instance normalization: The missing ingredient for fast stylization. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., and Vedaldi, A. (2013, January 23\u201328). Describing textures in the wild. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.","DOI":"10.1109\/CVPR.2014.461"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Tyle\u010dek, R., and \u0160\u00e1ra, R. (2013, January 3\u20136). Spatial pattern templates for recognition of objects with regular structure. Proceedings of the German Conference on Pattern Recognition, Saarbr\u00fccken, Germany.","DOI":"10.1007\/978-3-642-40602-7_39"},{"key":"ref_19","unstructured":"Karras, T., Aila, T., Laine, S., and Lehtine, J. (2017). Progressive growing of gans for improved quality, stability, and variation. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1452","DOI":"10.1109\/TPAMI.2017.2723009","article-title":"Places: A 10 Million Image Database for Scene Recognition","volume":"40","author":"Zhou","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Kim, P. (2017). Convolutional neural network. MATLAB Deep Learning, Apress.","DOI":"10.1007\/978-1-4842-2845-6"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1145\/3422622","article-title":"Generative adversarial networks","volume":"63","author":"Goodfellow","year":"2020","journal-title":"Commun. ACM"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., and Huang, T.S. (2018, January 18\u201323). Generative image inpainting with contextual attention. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00577"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Liu, G., Reda, F.A., Shih, K.J., Wang, T.C., Tao, A., and Catanzaro, B. (2018, January 8\u201314). Image inpainting for irregular holes using partial convolutions. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01252-6_6"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Zheng, C., Cham, T.J., and Cai, J. (2019, January 15\u201320). Pluralistic image completion. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00153"},{"key":"ref_26","first-page":"33","article-title":"Pyramid methods in image processing","volume":"29","author":"Adelson","year":"1984","journal-title":"RCA Eng."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Shocher, A., Gandelsman, Y., Mosseri, I., Yarom, M., Irani, M., Freeman, W.T., and Dekel, T. (2020, January 14\u201319). Semantic pyramid for image generation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00748"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Liu, Z., Luo, P., Wang, X., and Tang, X. (2015, January 7\u201313). Deep learning face attributes in the wild. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.425"},{"key":"ref_29","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_30","unstructured":"Dillon, J.V., Langmore, I., Tran, D., Brevdo, E., Vasudevan, S., Moore, D., and Saurous, R.A. (2017). Tensorflow distributions. arXiv."},{"key":"ref_31","first-page":"8026","article-title":"Pytorch: An imperative style, high-performance deep learning library","volume":"32","author":"Paszke","year":"2019","journal-title":"Adv. Neural Inf. Processing Syst."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2088","DOI":"10.4249\/scholarpedia.2088","article-title":"Signal-to-noise ratio","volume":"1","author":"Johnson","year":"2006","journal-title":"Scholarpedia"},{"key":"ref_33","unstructured":"Wang, Z., Simoncelli, E.P., and Bovik, A.C. (2003, January 9\u201312). Multiscale structural similarity for image quality assessment. Proceedings of the Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, Pacific Grove, CA, USA."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/13\/2\/71\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:12:32Z","timestamp":1760134352000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/13\/2\/71"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,1]]},"references-count":33,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2022,2]]}},"alternative-id":["info13020071"],"URL":"https:\/\/doi.org\/10.3390\/info13020071","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,2,1]]}}}