{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T03:00:49Z","timestamp":1760151649380,"version":"build-2065373602"},"reference-count":44,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2022,4,8]],"date-time":"2022-04-08T00:00:00Z","timestamp":1649376000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Shaanxi Natural Science Foundation","award":["No. 2022JM-335"],"award-info":[{"award-number":["No. 2022JM-335"]}]},{"name":"National Natural Science Foundation of China under Grant","award":["No. 61573232"],"award-info":[{"award-number":["No. 61573232"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Most existing image inpainting methods have achieved remarkable progress in small image defects. However, repairing large missing regions with insufficient context information is still an intractable problem. In this paper, a Multi-stage Feature Reasoning Generative Adversarial Network to gradually restore irregular holes is proposed. Specifically, dynamic partial convolution is used to adaptively adjust the restoration proportion during inpainting progress, which strengthens the correlation between valid and invalid pixels. In the decoding phase, the statistical natures of features in the masked areas differentiate from those of unmasked areas. To this end, a novel decoder is designed which not only dynamically assigns a scaling factor and bias on per feature point basis using point-wise normalization, but also utilizes skip connections to solve the problem of information loss between the codec network layers. Moreover, in order to eliminate gradient vanishing and increase the reasoning times, a hybrid weighted merging method consisting of a hard weight map and a soft weight map is proposed to ensemble the feature maps generated during the whole reconstruction process. Experiments on CelebA, Places2, and Paris StreetView show that the proposed model generates results with a PSNR improvement of 0.3 dB to 1.2 dB compared to other methods.<\/jats:p>","DOI":"10.3390\/s22082854","type":"journal-article","created":{"date-parts":[[2022,4,9]],"date-time":"2022-04-09T05:13:08Z","timestamp":1649481188000},"page":"2854","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Semantic Image Inpainting with Multi-Stage Feature Reasoning Generative Adversarial Network"],"prefix":"10.3390","volume":"22","author":[{"given":"Guangyao","family":"Li","sequence":"first","affiliation":[{"name":"School of Computer Science, Shaanxi Normal University, Xi\u2019an 710062, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1606-1671","authenticated-orcid":false,"given":"Liangfu","family":"Li","sequence":"additional","affiliation":[{"name":"School of Computer Science, Shaanxi Normal University, Xi\u2019an 710062, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yingdan","family":"Pu","sequence":"additional","affiliation":[{"name":"School of Computer Science, Shaanxi Normal University, Xi\u2019an 710062, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nan","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Computer Science, Shaanxi Normal University, Xi\u2019an 710062, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xi","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Computer Science, Shaanxi Normal University, Xi\u2019an 710062, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,4,8]]},"reference":[{"key":"ref_1","first-page":"7716","article-title":"Adversarial scene editing: Automatic object removal from weak supervision","volume":"31","author":"Shetty","year":"2018","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_2","unstructured":"Song, L., Cao, J., Song, L., Hu, Y., and He, R. (February, January 27). Geometry-aware face completion and editing. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA."},{"key":"ref_3","unstructured":"Armi, L., and Fekri-Ershad, S. (2019). Texture image analysis and texture classification methods\u2014A review. arXiv."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1145\/1531326.1531330","article-title":"PatchMatch: A randomized correspondence algorithm for structural image editing","volume":"28","author":"Barnes","year":"2009","journal-title":"ACM Trans. Graph."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1705","DOI":"10.1109\/TIP.2018.2880681","article-title":"Image inpainting using nonlocal texture matching and nonlinear filtering","volume":"28","author":"Ding","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Lee, J.H., Choi, I., and Kim, M.H. (2016, January 27\u201330). Laplacian patch-based image synthesis. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.298"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1200","DOI":"10.1109\/TIP.2004.833105","article-title":"Region filling and object removal by exemplar-based image inpainting","volume":"13","author":"Criminisi","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Sagong, M.c., Shin, Y.g., Kim, S.w., Park, S., and Ko, S.j. (2019, January 15\u201319). Pepsi: Fast image inpainting with parallel decoding network. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01162"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Nazeri, K., Ng, E., Joseph, T., Qureshi, F., and Ebrahimi, M. (2019, January 15\u201319). Edgeconnect: Structure guided image inpainting using edge prediction. Proceedings of the IEEE\/CVF International Conference on Computer Vision Workshops, Long Beach, CA, USA.","DOI":"10.1109\/ICCVW.2019.00408"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Li, C., He, K., Liu, K., and Ma, X. (2020). Image Inpainting Using Two-Stage Loss Function and Global and Local Markovian Discriminators. Sensors, 20.","DOI":"10.3390\/s20216193"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Yeh, R.A., Chen, C., Yian Lim, T., Schwing, A.G., Hasegawa-Johnson, M., and Do, M.N. (2017, January 21\u201326). Semantic image inpainting with deep generative models. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.728"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., and Huang, T.S. (2019, January 15\u201319). Free-form image inpainting with gated convolution. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Long Beach, CA, USA.","DOI":"10.1109\/ICCV.2019.00457"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., and Huang, T.S. (2018, January 18\u201323). Generative image inpainting with contextual attention. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00577"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zhang, H., Hu, Z., Luo, C., Zuo, W., and Wang, M. (2018, January 22\u201326). Semantic image inpainting with progressive generative networks. Proceedings of the 26th ACM international conference on Multimedia, Seoul, Korea.","DOI":"10.1145\/3240508.3240625"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Li, J., Wang, N., Zhang, L., Du, B., and Tao, D. (2020, January 14\u201319). Recurrent feature reasoning for image inpainting. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00778"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Isola, P., Zhu, J.Y., Zhou, T., and Efros, A.A. (2017, January 21\u201326). Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.632"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2185520.2185578","article-title":"Image melding: Combining inconsistent images using patch-based synthesis","volume":"31","author":"Darabi","year":"2012","journal-title":"ACM Trans. Graph."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1145\/1276377.1276382","article-title":"Scene completion using millions of photographs","volume":"26","author":"Hays","year":"2007","journal-title":"ACM Trans. Graph."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Liu, H., Jiang, B., Xiao, Y., and Yang, C. (2019, January 15\u201319). Coherent semantic attention for image inpainting. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Long Beach, CA, USA.","DOI":"10.1109\/ICCV.2019.00427"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Yi, Z., Tang, Q., Azizi, S., Jang, D., and Xu, Z. (2020, January 14\u201319). Contextual residual aggregation for ultra high-resolution image inpainting. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00753"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"He, X., and Yin, Y. (2021). Non-Local and Multi-Scale Mechanisms for Image Inpainting. Sensors, 21.","DOI":"10.3390\/s21093281"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Dong, H., Liang, X., Zhang, Y., Zhang, X., Shen, X., Xie, Z., Wu, B., and Yin, J. (2020, January 14\u201319). Fashion editing with adversarial parsing learning. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00814"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Liu, H., Wan, Z., Huang, W., Song, Y., Han, X., and Liao, J. (2021, January 20\u201325). PD-GAN: Probabilistic Diverse GAN for Image Inpainting. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00925"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Suvorov, R., Logacheva, E., Mashikhin, A., Remizova, A., Ashukha, A., Silvestrov, A., Kong, N., Goka, H., Park, K., and Lempitsky, V. (2022, January 4\u20138). Resolution-robust Large Mask Inpainting with Fourier Convolutions. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.","DOI":"10.1109\/WACV51458.2022.00323"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"4855","DOI":"10.1109\/TIP.2021.3076310","article-title":"Image Inpainting by End-to-End Cascaded Refinement with Mask Awareness","volume":"30","author":"Zhu","year":"2021","journal-title":"IEEE Trans. Image Process."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Yang, S., Huang, R., and Han, F. (2021). Progressively Inpainting Images Based on a Forked-Then-Fused Decoder Network. Sensors, 21.","DOI":"10.3390\/s21196336"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., and Efros, A.A. (2016, January 27\u201330). Context encoders: Feature learning by inpainting. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.278"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Zhao, L., Mo, Q., Lin, S., Wang, Z., Zuo, Z., Chen, H., Xing, W., and Lu, D. (2020, January 14\u201319). Uctgan: Diverse image inpainting based on unsupervised cross-space translation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00578"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Yan, Z., Li, X., Li, M., Zuo, W., and Shan, S. (2018, January 8\u201314). Shift-net: Image inpainting via deep feature rearrangement. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01264-9_1"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Zeng, Y., Fu, J., Chao, H., and Guo, B. (2019, January 15\u201319). Learning pyramid-context encoder network for high-quality image inpainting. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00158"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Liu, G., Reda, F.A., Shih, K.J., Wang, T.C., Tao, A., and Catanzaro, B. (2018, January 8\u201314). Image inpainting for irregular holes using partial convolutions. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01252-6_6"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3072959.3073659","article-title":"Globally and locally consistent image completion","volume":"36","author":"Iizuka","year":"2017","journal-title":"ACM Trans. Graph."},{"key":"ref_33","unstructured":"Yu, T., Guo, Z., Jin, X., Wu, S., Chen, Z., Li, W., Zhang, Z., and Liu, S. (2020, January 7\u201312). Region normalization for image inpainting. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA."},{"key":"ref_34","unstructured":"Song, Y., Yang, C., Shen, Y., Wang, P., Huang, Q., and Kuo, C.C.J. (2018). Spg-net: Segmentation prediction and guidance network for image inpainting. arXiv."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Xiong, W., Yu, J., Lin, Z., Yang, J., Lu, X., Barnes, C., and Luo, J. (2019, January 15\u201319). Foreground-aware image inpainting. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00599"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Ren, Y., Yu, X., Zhang, R., Li, T.H., Liu, S., and Li, G. (2019, January 15\u201319). Structureflow: Image inpainting via structure-aware appearance flow. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Long Beach, CA, USA.","DOI":"10.1109\/ICCV.2019.00027"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Li, J., He, F., Zhang, L., Du, B., and Tao, D. (2019, January 15\u201319). Progressive reconstruction of visual structure for image inpainting. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Long Beach, CA, USA.","DOI":"10.1109\/ICCV.2019.00606"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Guo, Z., Chen, Z., Yu, T., Chen, J., and Liu, S. (2019, January 21\u201325). Progressive image inpainting with full-resolution residual network. Proceedings of the 27th Acm International Conference on Multimedia, Nice, France.","DOI":"10.1145\/3343031.3351022"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Gatys, L.A., Ecker, A.S., and Bethge, M. (2015). A neural algorithm of artistic style. arXiv.","DOI":"10.1167\/16.12.326"},{"key":"ref_40","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1452","DOI":"10.1109\/TPAMI.2017.2723009","article-title":"Places: A 10 million image database for scene recognition","volume":"40","author":"Zhou","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Liu, Z., Luo, P., Wang, X., and Tang, X. (2015, January 7\u201313). Deep learning face attributes in the wild. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.425"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1145\/2185520.2185597","article-title":"What makes paris look like paris?","volume":"31","author":"Doersch","year":"2012","journal-title":"ACM Trans. Graph."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Xie, C., Liu, S., Li, C., Cheng, M.M., Zuo, W., Liu, X., Wen, S., and Ding, E. (2019, January 15\u201319). Image inpainting with learnable bidirectional attention maps. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Long Beach, CA, USA.","DOI":"10.1109\/ICCV.2019.00895"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/8\/2854\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:50:16Z","timestamp":1760136616000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/8\/2854"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,8]]},"references-count":44,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2022,4]]}},"alternative-id":["s22082854"],"URL":"https:\/\/doi.org\/10.3390\/s22082854","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2022,4,8]]}}}