{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,8]],"date-time":"2026-02-08T01:19:42Z","timestamp":1770513582791,"version":"3.49.0"},"reference-count":32,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2020,10,30]],"date-time":"2020-10-30T00:00:00Z","timestamp":1604016000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61271326"],"award-info":[{"award-number":["61271326"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Image inpainting networks can produce visually reasonable results in the damaged regions. However, existing inpainting networks may fail to reconstruct the proper structures or tend to generate the results with color discrepancy. To solve this issue, this paper proposes an image inpainting approach using the proposed two-stage loss function. The loss function consists of different Gaussian kernels, which are utilized in different stages of network. The use of our two-stage loss function in coarse network helps to focus on the image structure, while the use of it in refinement network is helpful to restore the image details. Moreover, we proposed a global and local PatchGANs (GAN means generative adversarial network), named GL-PatchGANs, in which the global and local markovian discriminators were used to control the final results. This is beneficial to focus on the regions of interest (ROI) on different scales and tends to produce more realistic structural and textural details. We trained our network on three popular datasets on image inpainting separately, both Peak Signal to Noise ratio (PSNR) and Structural Similarity (SSIM) between our results, and ground truths on test images show that our network can achieve better performance compared with the recent works in most cases. Besides, the visual results on three datasets also show that our network can produce visual plausible results compared with the recent works.<\/jats:p>","DOI":"10.3390\/s20216193","type":"journal-article","created":{"date-parts":[[2020,10,30]],"date-time":"2020-10-30T09:29:32Z","timestamp":1604050172000},"page":"6193","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Image Inpainting Using Two-Stage Loss Function and Global and Local Markovian Discriminators"],"prefix":"10.3390","volume":"20","author":[{"given":"Chen","family":"Li","sequence":"first","affiliation":[{"name":"School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8529-7960","authenticated-orcid":false,"given":"Kai","family":"He","sequence":"additional","affiliation":[{"name":"School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China"}]},{"given":"Kun","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China"}]},{"given":"Xitao","family":"Ma","sequence":"additional","affiliation":[{"name":"School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China"}]}],"member":"1968","published-online":{"date-parts":[[2020,10,30]]},"reference":[{"key":"ref_1","unstructured":"Shetty, R.R., Fritz, M., and Schiele, B. (2018, January 3\u20138). Adversarial Scene Editing: Automatic Object Removal from Weak Supervision. Proceedings of the NIPS 2018: The 32nd Annual Conference on Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Song, L., Cao, J., Song, L., Hu, Y., and He, R. (February, January 27). Geometry-Aware Face Completion and Editing. Proceedings of the AAAI 2019: Thirty-Third AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.","DOI":"10.1609\/aaai.v33i01.33012506"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1145\/1531326.1531330","article-title":"PatchMatch: A randomized correspondence algorithm for structural image editing","volume":"28","author":"Barnes","year":"2009","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Bertalmio, M., Sapiro, G., Caselles, V., and Ballester, C. (2000, January 23\u201328). Image inpainting. Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, New Orleans, LA, USA.","DOI":"10.1145\/344779.344972"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Efros, A.A., and Freeman, W.T. (2001, January 12\u201317). Image quilting for texture synthesis and transfer. Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, Los Angeles, CA, USA.","DOI":"10.1145\/383259.383296"},{"key":"ref_6","first-page":"82","article-title":"Image melding: Combining inconsistent images using patch-based synthesis","volume":"31","author":"Darabi","year":"2012","journal-title":"Int. Conf. Comput. Graph. Interact. Tech."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1145\/1400181.1400202","article-title":"Scene completion using millions of photographs","volume":"51","author":"Hays","year":"2008","journal-title":"Commun. ACM"},{"key":"ref_8","first-page":"2672","article-title":"Generative Adversarial Nets","volume":"27","author":"Goodfellow","year":"2014","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1016\/j.ins.2020.06.023","article-title":"Cross-Attentional Bracket-shaped Convolutional Network for semantic image segmentation","volume":"539","author":"Hua","year":"2020","journal-title":"Inf. Sci."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1007\/s10489-010-0212-9","article-title":"A neural network based retrainable framework for robust object recognition with application to mobile robotics","volume":"35","author":"An","year":"2011","journal-title":"Appl. Intell."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1109\/72.822517","article-title":"On-Line Retrainable Neural Networks: Improving the Performance of Neural Networks in Image Analysis Problems","volume":"11","author":"Doulamis","year":"2000","journal-title":"Trans. Neur. Netw."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., and Efros, A.A. (2016, January 27\u201330). Context Encoders: Feature Learning by Inpainting. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.278"},{"key":"ref_13","first-page":"107","article-title":"Globally and locally consistent image completion","volume":"36","author":"Iizuka","year":"2017","journal-title":"Int. Conf. Comput. Graph. Interact. Tech."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Liu, G., Reda, F.A., Shih, K.J., Wang, T.C., Tao, A., and Catanzaro, B. (2018, January 8\u201314). Image Inpainting for Irregular Holes Using Partial Convolutions. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01252-6_6"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., and Huang, T.S. (2018, January 18\u201322). Generative Image Inpainting with Contextual Attention. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00577"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., and Huang, T. (2019, January 22). Free-Form Image Inpainting With Gated Convolution. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00457"},{"key":"ref_17","unstructured":"Nazeri, K., Ng, E., Joseph, T., Qureshi, F.Z., and Ebrahimi, M. (2019). EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning. arXiv."},{"key":"ref_18","unstructured":"Song, Y., Yang, C., Shen, Y., Wang, P., Huang, Q., and Kuo, C.C.J. (2018). SPG-Net: Segmentation Prediction and Guidance Network for Image Inpainting. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ren, Y., Yu, X., Zhang, R., Li, T.H., Liu, S., and Li, G. (2019, January 22). StructureFlow: Image Inpainting via Structure-Aware Appearance Flow. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00027"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Kupyn, O., Martyniuk, T., Wu, J., and Wang, Z. (2019, January 22). DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00897"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Xie, C., Liu, S., Li, C., Cheng, M.M., Zuo, W., Liu, X., Wen, S., and Ding, E. (2019, January 22). Image Inpainting with Learnable Bidirectional Attention Maps. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00895"},{"key":"ref_22","unstructured":"Yang, J., Qi, Z., and Shi, Y. (2020, January 7\u201312). Learning to Incorporate Structure Knowledge for Image Inpainting. Proceedings of the AAAI 2020: The Thirty-Fourth AAAI Conference on Artificial Intelligence, New York, NY, USA."},{"key":"ref_23","unstructured":"Yu, T., Guo, Z., Jin, X., Wu, S., Chen, Z., Li, W., Zhang, Z., and Liu, S. (2020, January 7\u201312). Region Normalization for Image Inpainting. Proceedings of the AAAI 2020: The Thirty-Fourth AAAI Conference on Artificial Intelligence, New York, NY, USA."},{"key":"ref_24","unstructured":"Karras, T., Aila, T., Laine, S., and Lehtinen, J. (May, January 30). Progressive Growing of GANs for Improved Quality, Stability, and Variation. Proceedings of the ICLR 2018: International Conference on Learning Representations 2018, Vancouver, BC, Canada."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Isola, P., Zhu, J.Y., Zhou, T., and Efros, A.A. (2017, January 21\u201326). Image-to-Image Translation with Conditional Adversarial Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.632"},{"key":"ref_26","unstructured":"Yu, F., and Koltun, V. (2016, January 2\u20134). Multi-Scale Context Aggregation by Dilated Convolutions. Proceedings of the ICLR 2016: International Conference on Learning Representations 2016, San Juan, Puerto Rico."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1452","DOI":"10.1109\/TPAMI.2017.2723009","article-title":"Places: A 10 Million Image Database for Scene Recognition","volume":"40","author":"Zhou","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_28","unstructured":"Kingma, D.P., and Ba, J.L. (2015, January 7\u20139). Adam: A Method for Stochastic Optimization. Proceedings of the ICLR 2015: International Conference on Learning Representations 2015, San Diego, CA, USA."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1200","DOI":"10.1109\/TIP.2004.833105","article-title":"Region filling and object removal by exemplar-based image inpainting","volume":"13","author":"Criminisi","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_30","unstructured":"Mao, X.J., Shen, C., and Yang, Y.B. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. Proceedings of the NIPS\u201916: Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Spain, 5\u201310 December 2016."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Zoran, D., and Weiss, Y. (2011, January 6\u201313). From learning models of natural image patches to whole image restoration. Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain.","DOI":"10.1109\/ICCV.2011.6126278"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Liu, H., Jiang, B., Xiao, Y., and Yang, C. (2019, January 22). Coherent Semantic Attention for Image Inpainting. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00427"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/21\/6193\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:27:05Z","timestamp":1760178425000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/21\/6193"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,30]]},"references-count":32,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2020,11]]}},"alternative-id":["s20216193"],"URL":"https:\/\/doi.org\/10.3390\/s20216193","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,10,30]]}}}