{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,10]],"date-time":"2026-03-10T14:51:59Z","timestamp":1773154319893,"version":"3.50.1"},"reference-count":45,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2024,3,9]],"date-time":"2024-03-09T00:00:00Z","timestamp":1709942400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,3,9]],"date-time":"2024-03-09T00:00:00Z","timestamp":1709942400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"NSFC of China","award":["U1504610"],"award-info":[{"award-number":["U1504610"]}]},{"name":"NSFC of China","award":["61971339"],"award-info":[{"award-number":["61971339"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Process Lett"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In recent years, facial prior has been widely applied to enhance the quality of super-resolution (SR) facial images in face super-resolution (FSR) methods based on deep learning. However, most of the existing facial prior-based FSR methods have insufficient attention to local texture details, which can cause the generated SR facial images with overly smooth and unrealistic texture details, and show obvious artifacts under large magnification. With the help of GAN prior, recent advances can produce excellent results in terms of fidelity and realness. A generative framework for FSR is proposed in this work, which combines GAN and facial prior, termed CFGPFSR. Firstly, we pre-train a face StyleGAN2 and a face parsing network (FPN) that can generate decent parsing maps, in which the proposed CFGPFSR exploits rich and varied priors encapsulated in the face StyleGAN2 (GAN prior) and face parsing maps extracted from the FPN (facial prior) for FSR. Moreover, we introduce the Channel-Split Spatial Feature Transform (CS-SFT) method to further improve FSR performance. GAN and facial priors are introduced into the FSR process through the designed CS-SFT layers so that SR facial images obtain a promising balance between fidelity and realness. Unlike GAN inversion methods which necessitate costly image optimization at runtime, the proposed CFGPFSR can jointly recover facial details by only utilizing one forward pass. Experimental results on synthetic and real images indicate that the proposed CFGPFSR obtains remarkable performance in 16\u2009\u00d7\u2009SR task, and some of its metrics such as peak signal to noise ratio (PSNR) and structural similarity (SSIM) are higher than that of the comparison methods. Meanwhile, it shows impressive results in reconstructing high-quality facial images.<\/jats:p>","DOI":"10.1007\/s11063-024-11562-8","type":"journal-article","created":{"date-parts":[[2024,3,9]],"date-time":"2024-03-09T16:01:42Z","timestamp":1710000102000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["CFGPFSR: A Generative Method Combining Facial and GAN Priors for Face Super-Resolution"],"prefix":"10.1007","volume":"56","author":[{"given":"Jinbo","family":"Liu","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhonghua","family":"Liu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Weihua","family":"Ou","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kaibing","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yong","family":"Liu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,3,9]]},"reference":[{"key":"11562_CR1","doi-asserted-by":"publisher","first-page":"106663","DOI":"10.1016\/j.knosys.2020.106663","volume":"213","author":"Y Dun","year":"2021","unstructured":"Dun Y, Da Z, Yang S et al (2021) Kernel-attended residual network for single image super-resolution. Knowl-Based Syst 213:106663","journal-title":"Knowl-Based Syst"},{"key":"11562_CR2","doi-asserted-by":"crossref","unstructured":"Shi W, Caballero J, Husz\u00e1r F, et al. (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp. 1874\u20131883","DOI":"10.1109\/CVPR.2016.207"},{"key":"11562_CR3","doi-asserted-by":"crossref","unstructured":"Chen Y, Tai Y, Liu X, et al. (2018) Fsrnet: End-to-end learning face super-resolution with facial priors. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR), pp. 2492\u20132501","DOI":"10.1109\/CVPR.2018.00264"},{"key":"11562_CR4","doi-asserted-by":"crossref","unstructured":"Yu X, Fernando B, Ghanem B, et al. (2018) Face super-resolution guided by facial component heatmaps. In: European conference on computer vision (ECCV), pp. 217\u2013233","DOI":"10.1007\/978-3-030-01240-3_14"},{"key":"11562_CR5","doi-asserted-by":"publisher","first-page":"468","DOI":"10.1109\/TMM.2020.2984092","volume":"23","author":"M Li","year":"2021","unstructured":"Li M, Zhang Z et al (2021) Learning face image super-resolution through facial semantic attribute transformation and self-attentive structure enhancement. IEEE Trans Multimedia 23:468\u2013483","journal-title":"IEEE Trans Multimedia"},{"key":"11562_CR6","doi-asserted-by":"crossref","unstructured":"MaC, Jiang Z, Rao Y, et al. (2020) Deep face super-resolution with iterative collaboration between attentive recovery and landmark estimation. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR), pp. 5569\u20135578","DOI":"10.1109\/CVPR42600.2020.00561"},{"issue":"7","key":"11562_CR7","first-page":"12476","volume":"34","author":"J Xin","year":"2020","unstructured":"Xin J, Wang N, Jiang X et al (2020) Facial attribute capsules for noise face super resolution. Proc Assoc Adv Artif Intell 34(7):12476\u201312483","journal-title":"Proc Assoc Adv Artif Intell"},{"issue":"11","key":"11562_CR8","first-page":"2926","volume":"42","author":"X Yu","year":"2020","unstructured":"Yu X, Fernando B, Hartley R et al (2020) Semantic face hallucination: Super-resolving very low-resolution face images with supplementary attributes. IEEE Trans Pattern Anal Mach Intell 42(11):2926\u20132943","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"11562_CR9","doi-asserted-by":"crossref","unstructured":"Gu J, Shen Y, Zhou B (2020) Image processing using multi-code gan prior. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR), pp. 3009\u20133018","DOI":"10.1109\/CVPR42600.2020.00308"},{"key":"11562_CR10","doi-asserted-by":"crossref","unstructured":"Menon S, Damian A, Hu S, et al. (2020) Pulse: Self-supervised photo upsampling via latent space exploration of generative models. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR), pp. 2434\u20132442","DOI":"10.1109\/CVPR42600.2020.00251"},{"key":"11562_CR11","doi-asserted-by":"crossref","unstructured":"Pan X, Zhan X, et al. (2020) Exploiting deep generative prior for versatile image restoration and manipulation. In: European conference on computer vision (ECCV), pp. 23\u201328","DOI":"10.1007\/978-3-030-58536-5_16"},{"key":"11562_CR12","doi-asserted-by":"crossref","unstructured":"Chan KCK, Wang X, Xu X, et al. (2021) Glean: Generative latent bank for large-factor image super-resolution. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR), pp. 14245\u201314254","DOI":"10.1109\/CVPR46437.2021.01402"},{"key":"11562_CR13","doi-asserted-by":"crossref","unstructured":"Karras T, Laine S, Aittala M, et al. (2020) Analyzing and improving the image quality of stylegan. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR), pp. 8110\u20138119","DOI":"10.1109\/CVPR42600.2020.00813"},{"key":"11562_CR14","doi-asserted-by":"crossref","unstructured":"Wang X, Li Y, et al. (2021) Towards real-world blind face restoration with generative facial prior. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR), pp. 9168\u20139178","DOI":"10.1109\/CVPR46437.2021.00905"},{"key":"11562_CR15","doi-asserted-by":"crossref","unstructured":"Yang T, Ren P, Xie X, et al. (2021) Gan prior embedded network for blind face restoration in the wild. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR), pp. 672\u2013681","DOI":"10.1109\/CVPR46437.2021.00073"},{"key":"11562_CR16","doi-asserted-by":"crossref","unstructured":"Baker S, Kanade T (2000) Hallucinating faces. In: Proceedings fourth IEEE international conference on automatic face and gesture recognition (Cat. No. PR00580), pp. 83\u201388","DOI":"10.1109\/AFGR.2000.840616"},{"issue":"3","key":"11562_CR17","doi-asserted-by":"publisher","first-page":"425","DOI":"10.1109\/TSMCC.2005.848171","volume":"35","author":"X Wang","year":"2005","unstructured":"Wang X, Tang X (2005) Hallucinating face by eigentransformation. IEEE Trans Syst Man Cyber Part C 35(3):425\u2013434","journal-title":"IEEE Trans Syst Man Cyber Part C"},{"issue":"1","key":"11562_CR18","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1007\/s11263-006-0029-5","volume":"75","author":"C Liu","year":"2007","unstructured":"Liu C, Shum H-Y, Freeman WT (2007) Face hallucination: Theory and practice. Int J Comput Vis (IJCV) 75(1):115\u2013134","journal-title":"Int J Comput Vis (IJCV)"},{"issue":"6","key":"11562_CR19","doi-asserted-by":"publisher","first-page":"2224","DOI":"10.1016\/j.patcog.2009.12.019","volume":"43","author":"X Ma","year":"2010","unstructured":"Ma X, Zhang J, Qi C (2010) Hallucinating face by position-patch. Pattern Recogn (PR) 43(6):2224\u20132236","journal-title":"Pattern Recogn (PR)"},{"issue":"6","key":"11562_CR20","doi-asserted-by":"publisher","first-page":"2224","DOI":"10.1016\/j.patcog.2009.12.019","volume":"43","author":"X Ma","year":"2010","unstructured":"Ma X, Zhang J, Qi C (2010) Hallucinating face by position-patch. Pattern Recogn 43(6):2224\u20132236","journal-title":"Pattern Recogn"},{"key":"11562_CR21","doi-asserted-by":"crossref","unstructured":"SongY, Zhang J, He S, et al. (2017) Learning to hallucinate face images via component generation and enhancement. In: International joint conference on artificial intelligence (IJCAI), pp. 4537\u20134543","DOI":"10.24963\/ijcai.2017\/633"},{"key":"11562_CR22","doi-asserted-by":"crossref","unstructured":"Jing Y, Yang Y, Wang X, et al. (2021) Amalgamating knowledge from heterogeneous graph neural networks. In: IEEE\/CVF conference on computer vision and pattern recognition (CVPR), pp. 15704\u201315713","DOI":"10.1109\/CVPR46437.2021.01545"},{"key":"11562_CR23","doi-asserted-by":"crossref","unstructured":"Q. Cao, L. Lin, Y. Shi, et al. Attention-aware face hallucination via deep reinforcement learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 1656\u20131664","DOI":"10.1109\/CVPR.2017.180"},{"key":"11562_CR24","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1016\/j.neucom.2019.09.079","volume":"376","author":"X Chen","year":"2020","unstructured":"Chen X, Wang X, Yao Lu et al (2020) Rbpnet: An asymptotic residual back-projection network for super-resolution of very low-resolution face image. Neurocomputing 376:119\u2013127","journal-title":"Neurocomputing"},{"issue":"1","key":"11562_CR25","doi-asserted-by":"publisher","first-page":"378","DOI":"10.1109\/TNNLS.2020.3027849","volume":"33","author":"K Jiang","year":"2022","unstructured":"Jiang K, Wang Z, Yi P et al (2022) Dual-path deep fusion network for face image hallucination. IEEE Trans Neural Netw Learn Syst 33(1):378\u2013391","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"key":"11562_CR26","doi-asserted-by":"publisher","first-page":"1219","DOI":"10.1109\/TIP.2020.3043093","volume":"30","author":"C Chen","year":"2021","unstructured":"Chen C, Gong D, Wang H et al (2021) Learning spatial attention for face super-resolution. IEEE Trans Image Process 30:1219\u20131231","journal-title":"IEEE Trans Image Process"},{"key":"11562_CR27","doi-asserted-by":"crossref","unstructured":"Jing Y, Yang Y, Wang X, et al. (2021) Meta-aggregator: learning to aggregate for 1-bit graph neural networks. In: IEEE\/CVF international conference on computer vision (ICCV), pp. 5281\u20135290","DOI":"10.1109\/ICCV48922.2021.00525"},{"key":"11562_CR28","doi-asserted-by":"crossref","unstructured":"Jing Y, Yuan C, Ju L, et al. (2023) Deep graph reprogramming. In: IEEE\/CVF conference on computer vision and pattern recognition (CVPR), pp. 24345\u201324354","DOI":"10.1109\/CVPR52729.2023.02332"},{"key":"11562_CR29","doi-asserted-by":"publisher","first-page":"1505","DOI":"10.1109\/TMM.2023.3240880","volume":"25","author":"G Gao","year":"2023","unstructured":"Gao G, Tang L, Fei Wu et al (2023) Jdsr-gan: constructing an efficient joint learning network for masked face super-resolution. IEEE Trans Multimed 25:1505\u20131512","journal-title":"IEEE Trans Multimed"},{"key":"11562_CR30","doi-asserted-by":"publisher","first-page":"106987","DOI":"10.1016\/j.knosys.2021.106987","volume":"222","author":"H Wang","year":"2021","unstructured":"Wang H, Qian Hu, Chengdong Wu et al (2021) Dclnet: Dual closed-loop networks for face super-resolution. Knowl-Based Syst 222:106987","journal-title":"Knowl-Based Syst"},{"key":"11562_CR31","doi-asserted-by":"crossref","unstructured":"Karras T, Laine S, Aila T (2019) A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR), pp. 4401\u20134410","DOI":"10.1109\/CVPR.2019.00453"},{"key":"11562_CR32","doi-asserted-by":"crossref","unstructured":"Wang X, Yu K, Wu S, et al. (2018) Esrgan: Enhanced super-resolution generative adversarial networks. In: Proceedings of the European conference on computer vision (ECCV) workshops, Springer, pp. 63\u201379","DOI":"10.1007\/978-3-030-11021-5_5"},{"key":"11562_CR33","doi-asserted-by":"crossref","unstructured":"Wang X, Yu K, Dong C, et al. (2018) Recovering realistic texture in image super-resolution by deep spatial feature transform. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR), pp. 606\u2013615","DOI":"10.1109\/CVPR.2018.00070"},{"key":"11562_CR34","doi-asserted-by":"crossref","unstructured":"Li X, Chen C, Zhou S, et al. (2020) Blind face restoration via deep multi-scale component dictionaries. In: European conference on computer vision (ECCV), pp. 399\u2013415","DOI":"10.1007\/978-3-030-58545-7_23"},{"key":"11562_CR35","doi-asserted-by":"crossref","unstructured":"Park T, Liu M-Y, et al. (2019) Semantic image synthesis with spatially-adaptive normalization. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR), pp. 2332\u20132341","DOI":"10.1109\/CVPR.2019.00244"},{"key":"11562_CR36","doi-asserted-by":"crossref","unstructured":"Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. In: European conference on computer vision (ECCV), Springer, pp. 694\u2013711","DOI":"10.1007\/978-3-319-46475-6_43"},{"key":"11562_CR37","unstructured":"Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International conference on learning representations (ICLR), pp. 1\u201314"},{"key":"11562_CR38","unstructured":"Goodfellow I, Pouget-Abadie J, Mirza M, et al. (2014) Generative adversarial nets. In: Proceedings of the 27th international conference on neural information processing systems, pp. 2672\u20132680"},{"key":"11562_CR39","doi-asserted-by":"crossref","unstructured":"Zhang R, Isola P, Efros AA, et al. (2018) The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp. 586\u2013595","DOI":"10.1109\/CVPR.2018.00068"},{"key":"11562_CR40","unstructured":"Heusel M, Ramsauer H, Unterthiner T, et al. (2017) Gans trained by a two time-scale update rule converge to a local nash equilibrium. In: Proceedings of the 31st international conference on neural information processing systems, Vol. 30. pp. 6629\u20136640"},{"issue":"3","key":"11562_CR41","doi-asserted-by":"publisher","first-page":"209","DOI":"10.1109\/LSP.2012.2227726","volume":"20","author":"A Mittal","year":"2012","unstructured":"Mittal A, Soundararajan R, Bovik AC (2012) Making a \u201ccompletely blind\u201d image quality analyzer. IEEE Signal Process Lett 20(3):209\u2013212","journal-title":"IEEE Signal Process Lett"},{"key":"11562_CR42","doi-asserted-by":"crossref","unstructured":"Deng J, Guo J, Xue N, et al. (2019) Arcface: Additive angular margin loss for deep face recognition. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR), pp. 4690\u20134699","DOI":"10.1109\/CVPR.2019.00482"},{"key":"11562_CR43","unstructured":"Loshchilov I, Hutter F (2016) SGDR: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983"},{"key":"11562_CR44","unstructured":"Kingma D, Ba J (2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980"},{"key":"11562_CR45","doi-asserted-by":"publisher","first-page":"1587","DOI":"10.1049\/iet-ipr.2019.0129","volume":"14","author":"RA Dehkordi","year":"2020","unstructured":"Dehkordi RA et al (2020) Single image super-resolution based on sparse representation using dictionaries trained with input image patches. IET Image Process 14:1587\u20131593","journal-title":"IET Image Process"}],"container-title":["Neural Processing Letters"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11063-024-11562-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11063-024-11562-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11063-024-11562-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,16]],"date-time":"2024-05-16T20:34:14Z","timestamp":1715891654000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11063-024-11562-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,9]]},"references-count":45,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,4]]}},"alternative-id":["11562"],"URL":"https:\/\/doi.org\/10.1007\/s11063-024-11562-8","relation":{},"ISSN":["1573-773X"],"issn-type":[{"value":"1573-773X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,9]]},"assertion":[{"value":"11 February 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 March 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}},{"value":"The authors declare that there are no conflicts of interest regarding the publication of this paper.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflicts of Interest"}}],"article-number":"97"}}