{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,27]],"date-time":"2025-09-27T21:59:54Z","timestamp":1759010394310,"version":"3.41.2"},"reference-count":47,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2022,11,24]],"date-time":"2022-11-24T00:00:00Z","timestamp":1669248000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Comput. Sci."],"abstract":"<jats:p>Extreme face super-resolution (FSR), that is, improving the resolution of face images by an extreme scaling factor (often greater than \u00d78) has remained underexplored in the literature of low-level vision. Extreme FSR in the wild must address the challenges of both unpaired training data and unknown degradation factors. Inspired by the latest advances in image super-resolution (SR) and self-supervised learning (SSL), we propose a novel two-step approach to FSR by introducing a mid-resolution (MR) image as the stepping stone. In the first step, we leverage ideas from SSL-based SR reconstruction of medical images (e.g., MRI and ultrasound) to modeling the realistic degradation process of face images in the real world; in the second step, we extract the latent codes from MR images and interpolate them in a self-supervised manner to facilitate artifact-suppressed image reconstruction. Our two-step extreme FSR can be interpreted as the combination of existing self-supervised CycleGAN (step 1) and StyleGAN (step 2) that overcomes the barrier of critical resolution in face recognition. Extensive experimental results have shown that our two-step approach can significantly outperform existing state-of-the-art FSR techniques, including FSRGAN, Bulat's method, and PULSE, especially for large scaling factors such as 64.<\/jats:p>","DOI":"10.3389\/fcomp.2022.1037435","type":"journal-article","created":{"date-parts":[[2022,11,24]],"date-time":"2022-11-24T13:05:03Z","timestamp":1669295103000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Toward extreme face super-resolution in the wild: A self-supervised learning approach"],"prefix":"10.3389","volume":"4","author":[{"given":"Ahmed","family":"Cheikh Sidiya","sequence":"first","affiliation":[]},{"given":"Xin","family":"Li","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2022,11,24]]},"reference":[{"key":"B1","first-page":"4432","article-title":"\u201cImage2stylegan: how to embed images into the stylegan latent space?\u201d","author":"Abdal","year":"2019","journal-title":"Proceedings of the IEEE International Conference on Computer Vision"},{"key":"B2","doi-asserted-by":"publisher","first-page":"308","DOI":"10.1016\/j.patcog.2019.01.032","article-title":"SSR2: sparse signal recovery for single-image super-resolution on faces with extreme low resolutions","volume":"90","author":"Abiantun","year":"2019","journal-title":"Pattern Recogn."},{"key":"B3","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1080\/09541449108406221","article-title":"Openface: a general-purpose face recognition library with mobile applications","volume":"6","author":"Amos","year":"2016","journal-title":"CMU Schl. Comput. Sci."},{"key":"B4","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1109\/TPAMI.2016.2644615","article-title":"Identification of spatially quantised tachistoscopic images of faces: how many pixels does it take to carry identity?","volume":"3","author":"Bachmann","year":"1991","journal-title":"Eur. J. Cogn. Psychol."},{"key":"B5","doi-asserted-by":"crossref","first-page":"2481","DOI":"10.1109\/TPAMI.2016.2644615","article-title":"SEGNet: a deep convolutional encoder-decoder architecture for image segmentation","volume":"39","author":"Badrinarayanan","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"B6","first-page":"185","article-title":"\u201cTo learn image super-resolution, use a GAN to learn how to do image degradation first,\u201d","author":"Bulat","year":"2018","journal-title":"Proceedings of the European Conference on Computer Vision (ECCV)"},{"key":"B7","first-page":"3086","article-title":"\u201cToward real-world single image super-resolution: a new benchmark and a new model,\u201d","author":"Cai","year":"2019","journal-title":"Proceedings of the IEEE International Conference on Computer Vision"},{"key":"B8","first-page":"2492","article-title":"\u201cFSRNet: end-to-end learning face super-resolution with facial priors,\u201d","author":"Chen","year":"2018","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B9","first-page":"139","article-title":"\u201cGenerative adversarial nets,\u201d","author":"Goodfellow","year":"2014","journal-title":"Communications of the ACM"},{"key":"B10","first-page":"1604","article-title":"\u201cBlind super-resolution with iterative kernel correction,\u201d","author":"Gu","year":"2019","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"year":"2017","author":"Heusel","key":"B11"},{"key":"B12","first-page":"1501","article-title":"\u201cArbitrary style transfer in real-time with adaptive instance normalization,\u201d","author":"Huang","year":"2017","journal-title":"Proceedings of the IEEE International Conference on Computer Vision"},{"key":"B13","doi-asserted-by":"publisher","first-page":"873","DOI":"10.1109\/TIP.2008.922421","article-title":"Generalized face super-resolution","volume":"17","author":"Jia","year":"2008","journal-title":"IEEE Trans. Image Process."},{"key":"B14","article-title":"The relativistic discriminator: a key element missing from standard GAN","author":"Jolicoeur-Martineau","year":"2018","journal-title":"arXiv [Preprint]."},{"key":"B15","first-page":"370","article-title":"\u201cComponent attention guided face super-resolution network: CAGFace,\u201d","author":"Kalarot","year":"2020","journal-title":"The IEEE Winter Conference on Applications of Computer Vision"},{"key":"B16","article-title":"Progressive growing of GANs for improved quality, stability, and variation","author":"Karras","year":"2017","journal-title":"arXiv [Preprint]."},{"key":"B17","first-page":"4401","article-title":"\u201cA style-based generator architecture for generative adversarial networks,\u201d","author":"Karras","year":"2019","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B18","first-page":"8110","article-title":"\u201cAnalyzing and improving the image quality of styleGAN,\u201d","author":"Karras","year":"2020","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B19","article-title":"Adam: a method for stochastic optimization","author":"Kingma","year":"2014","journal-title":"arXiv [Preprint]."},{"key":"B20","doi-asserted-by":"publisher","first-page":"2944","DOI":"10.1109\/TPAMI.2019.2917037","article-title":"Toward bridging the simulated-to-real gap: benchmarking super-resolution on real data","volume":"42","author":"K\u00f6hler","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B21","first-page":"4681","article-title":"\u201cPhoto-realistic single image super-resolution using a generative adversarial network,\u201d","author":"Ledig","year":"2017","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B22","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1007\/s11263-006-0029-5","article-title":"Face hallucination: theory and practice","volume":"75","author":"Liu","year":"2007","journal-title":"Int. J. Comput. Vis."},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-020-05687-9","article-title":"Perception consistency ultrasound image super-resolution via self-supervised CycleGAN","author":"Liu","year":"","journal-title":"Neural Comput. Appl."},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2021.3090866","article-title":"Self-supervised learning: generative or contrastive","author":"Liu","year":"","journal-title":"IEEE Trans. Knowledge Data Eng"},{"key":"B25","doi-asserted-by":"crossref","first-page":"3408","DOI":"10.1109\/ICCVW.2019.00423","article-title":"\u201cUnsupervised learning for real-world super-resolution,\u201d","author":"Lugmayr","year":"2019","journal-title":"2019 IEEE\/CVF International Conference on Computer Vision Workshop (ICCVW)"},{"key":"B26","first-page":"2437","article-title":"\u201cPulse: self-supervised photo upsampling via latent space exploration of generative models,\u201d","author":"Menon","year":"2020","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B27","doi-asserted-by":"publisher","first-page":"209","DOI":"10.1109\/LSP.2012.2227726","article-title":"Making a \u201ccompletely blind\u201d image quality analyzer","volume":"20","author":"Mittal","year":"2012","journal-title":"IEEE Signal Process. Lett."},{"key":"B28","article-title":"Spectral normalization for generative adversarial networks","author":"Miyato","year":"2018","journal-title":"arXiv [Preprint]."},{"key":"B29","first-page":"1121","article-title":"\u201cSelf-supervised multi-image super-resolution for push-frame satellite images,\u201d","author":"Nguyen","year":"2021","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B30","first-page":"8026","article-title":"\u201cPyTorch: an imperative style, high-performance deep learning library,\u201d","author":"Paszke","year":"2019","journal-title":"Advances in Neural Information Processing Systems, Vol. 32"},{"key":"B31","first-page":"9243","article-title":"\u201cInterpreting the latent space of gans for semantic face editing,\u201d","author":"Shen","year":"","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B32","doi-asserted-by":"publisher","first-page":"2004","DOI":"10.1109\/TPAMI.2020.3034267","article-title":"InterfaceGAN: interpreting the disentangled face representation learned by GANs","volume":"44","author":"Shen","year":"","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B33","first-page":"6142","article-title":"\u201cStyleRig: rigging styleGAN for 3D control over portrait images,\u201d","author":"Tewari","year":"2020","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58542-6_11","article-title":"StyleGAN2 distillation for feed-forward image manipulation","author":"Viazovetskyi","year":"2020","journal-title":"arXiv [Preprint]."},{"key":"B35","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1007\/s11263-013-0645-9","article-title":"A comprehensive survey to face hallucination","volume":"106","author":"Wang","year":"2014","journal-title":"Int. J. Comput. Vis."},{"key":"B36","first-page":"7794","article-title":"\u201cNon-local neural networks,\u201d","author":"Wang","year":"2018","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B37","article-title":"\u201cTowards real-world blind face restoration with generative facial prior,\u201d","author":"Wang","year":"2021","journal-title":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)"},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01318","article-title":"Unsupervised real-world image super resolution via domain-distance aware training","author":"Wei","year":"2020","journal-title":"arXiv [Preprint]."},{"key":"B39","article-title":"Improving inversion and generation diversity in styleGAN using a Gaussianized latent space","author":"Wulff","year":"2020","journal-title":"arXiv [Preprint]."},{"key":"B40","first-page":"5525","article-title":"\u201cWider face: a face detection benchmark,\u201d","author":"Yang","year":"2016","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B41","first-page":"7354","article-title":"\u201cSelf-attention generative adversarial networks,\u201d","author":"Zhang","year":"2019","journal-title":"International Conference on Machine Learning"},{"key":"B42","first-page":"586","article-title":"\u201cThe unreasonable effectiveness of deep features as a perceptual metric,\u201d","author":"Zhang","year":"","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.30","article-title":"S3FD: single shot scale-invariant face detector","author":"Zhang","year":"2017","journal-title":"arXiv [Preprint]."},{"key":"B44","first-page":"286","article-title":"\u201cImage super-resolution using very deep residual channel attention networks,\u201d","author":"Zhang","year":"","journal-title":"Proceedings of the European Conference on Computer Vision (ECCV)"},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-19797-0_35","article-title":"Self-supervised learning for real-world super-resolution from dual zoomed observations","author":"Zhang","year":"2022","journal-title":"arXiv [Preprint]."},{"key":"B46","doi-asserted-by":"publisher","first-page":"805","DOI":"10.1109\/TMI.2020.3037187","article-title":"Smore: a self-supervised anti-aliasing and super-resolution algorithm for mri using deep learning","volume":"40","author":"Zhao","year":"2020","journal-title":"IEEE Trans. Med. Imaging"},{"key":"B47","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58520-4_35","article-title":"In-domain gan inversion for real image editing","author":"Zhu","year":"2020","journal-title":"arXiv [Preprint]."}],"container-title":["Frontiers in Computer Science"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcomp.2022.1037435\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,24]],"date-time":"2022-11-24T13:05:19Z","timestamp":1669295119000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcomp.2022.1037435\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,24]]},"references-count":47,"alternative-id":["10.3389\/fcomp.2022.1037435"],"URL":"https:\/\/doi.org\/10.3389\/fcomp.2022.1037435","relation":{},"ISSN":["2624-9898"],"issn-type":[{"type":"electronic","value":"2624-9898"}],"subject":[],"published":{"date-parts":[[2022,11,24]]},"article-number":"1037435"}}