{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,4]],"date-time":"2026-07-04T02:11:25Z","timestamp":1783131085239,"version":"3.54.6"},"reference-count":69,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,9,23]],"date-time":"2024-09-23T00:00:00Z","timestamp":1727049600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:p>Real-valued convolutional neural networks (RV-CNNs) in the spatial domain have outperformed classical approaches in many image restoration tasks such as image denoising and super-resolution. Fourier analysis of the results produced by these spatial domain models reveals the limitations of these models in properly processing the full frequency spectrum. This lack of complete spectral information can result in missing textural and structural elements. To address this limitation, we explore the potential of complex-valued convolutional neural networks (CV-CNNs) for image restoration tasks. CV-CNNs have shown remarkable performance in tasks such as image classification and segmentation. However, CV-CNNs for image restoration problems in the frequency domain have not been fully investigated to address the aforementioned issues. Here, we propose several novel CV-CNN-based models equipped with complex-valued attention gates for image denoising and super-resolution in the frequency domains. We also show that our CV-CNN-based models outperform their real-valued counterparts for denoising super-resolution structured illumination microscopy (SR-SIM) and conventional image datasets. Furthermore, the experimental results show that our proposed CV-CNN-based models preserve the frequency spectrum better than their real-valued counterparts in the denoising task. Based on these findings, we conclude that CV-CNN-based methods provide a plausible and beneficial deep learning approach for image restoration in the frequency domain.<\/jats:p>","DOI":"10.3389\/frai.2024.1353873","type":"journal-article","created":{"date-parts":[[2024,9,23]],"date-time":"2024-09-23T08:34:53Z","timestamp":1727080493000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Image restoration in frequency space using complex-valued CNNs"],"prefix":"10.3389","volume":"7","author":[{"given":"Zafran Hussain","family":"Shah","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marcel","family":"M\u00fcller","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wolfgang","family":"H\u00fcbner","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Henning","family":"Ortkrass","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Barbara","family":"Hammer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas","family":"Huser","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wolfram","family":"Schenck","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2024,9,23]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"169","DOI":"10.4103\/2228-7477.112144","article-title":"Extracting, recognizing, and counting white blood cells from microscopic images by using complex-valued neural networks","volume":"2","author":"Akramifard","year":"2012","journal-title":"J. Med. Signals Sens"},{"key":"B2","doi-asserted-by":"publisher","first-page":"2169","DOI":"10.1038\/s41467-021-22377-9","article-title":"Two-dimensional TIRF-SIM-traction force microscopy (2d TIRF-SIM-TFM)","volume":"12","author":"Barbieri","year":"2021","journal-title":"Nat. Commun"},{"key":"B3","doi-asserted-by":"crossref","first-page":"2990","DOI":"10.1109\/ICASSP39728.2021.9413814","article-title":"\u201cComplex-valued vs. real-valued neural networks for classification perspectives: an example on non-circular data,\u201d","volume-title":"ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","author":"Barrachina","year":"2021"},{"key":"B4","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1007\/s11265-022-01793-0","article-title":"Comparison between equivalent architectures of complex-valued and real-valued neural networks-application on polarimetric sar image segmentation","volume":"95","author":"Barrachina","year":"2022","journal-title":"J. Signal Process. Syst"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2101.12249","article-title":"A survey of complex-valued neural networks","author":"Bassey","year":"2021","journal-title":"arXiv"},{"key":"B6","doi-asserted-by":"publisher","first-page":"1215","DOI":"10.1038\/s41592-019-0458-z","article-title":"Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction","volume":"16","author":"Belthangady","year":"2019","journal-title":"Nat. Methods"},{"key":"B7","doi-asserted-by":"publisher","first-page":"967","DOI":"10.1109\/78.127967","article-title":"On the complex backpropagation algorithm","volume":"40","author":"Benvenuto","year":"1992","journal-title":"IEEE Transact. Signal Process"},{"key":"B8","doi-asserted-by":"publisher","first-page":"700","DOI":"10.1109\/TCI.2021.3093788","article-title":"Accelerated phase shifting for structured illumination microscopy based on deep learning","volume":"7","author":"Chen","year":"2021","journal-title":"IEEE Transact. Comp. Imaging"},{"key":"B9","unstructured":"\u201cDeep complex networks,\u201d\n          \n          \n            \n              Chiheb\n              T.\n            \n            \n              Bilaniuk\n              O.\n            \n            \n              Serdyuk\n              D.\n            \n          \n          International Conference on Learning Representations\n          \n          2017"},{"key":"B10","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1007\/978-3-030-00129-2_4","article-title":"\u201cComplex fully convolutional neural networks for mr image reconstruction,\u201d","volume-title":"Machine Learning for Medical Image Reconstruction: First International Workshop, MLMIR 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Proceedings 1","author":"Dedmari","year":"2018"},{"key":"B11","doi-asserted-by":"publisher","first-page":"988","DOI":"10.1038\/nprot.2017.019","article-title":"Strategic and practical guidelines for successful structured illumination microscopy","volume":"12","author":"Demmerle","year":"2017","journal-title":"Nat. Protoc"},{"key":"B12","first-page":"640","article-title":"\u201cFading channel prediction based on complex-valued neural networks in frequency domain,\u201d","volume-title":"2013 International Symposium on Electromagnetic Theory","author":"Ding","year":"2013"},{"key":"B13","first-page":"184","article-title":"\u201cLearning a deep convolutional network for image super-resolution,\u201d","volume-title":"European Conference on Computer Vision","author":"Dong","year":"2014"},{"key":"B14","volume-title":"Statistical Signal Processing of Complex-Valued Data: The Theory of Improper and Noncircular","author":"Foreman","year":"2012"},{"key":"B15","doi-asserted-by":"publisher","first-page":"123391","DOI":"10.1016\/j.eswa.2024.123391","article-title":"Biologically inspired image invariance guided illuminant estimation using shallow and deep models","volume":"249","author":"Gao","year":"2024","journal-title":"Exp. Syst. Appl."},{"key":"B16","doi-asserted-by":"publisher","first-page":"330","DOI":"10.1109\/82.142037","article-title":"Complex domain backpropagation","volume":"39","author":"Georgiou","year":"1992","journal-title":"IEEE Transact. Circ. Syst"},{"key":"B17","volume-title":"Digital Image Processing","author":"Gonzalez","year":"2009"},{"key":"B18","first-page":"3128","article-title":"\u201cFast image super resolution via local regression,\u201d","volume-title":"Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012)","author":"Gu","year":"2012"},{"key":"B19","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1046\/j.1365-2818.2000.00710.x","article-title":"Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy","volume":"198","author":"Gustafsson","year":"2000","journal-title":"J. Microsc"},{"key":"B20","doi-asserted-by":"publisher","first-page":"4957","DOI":"10.1529\/biophysj.107.120345","article-title":"Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination","volume":"94","author":"Gustafsson","year":"2008","journal-title":"Biophys. J"},{"key":"B21","doi-asserted-by":"publisher","first-page":"649","DOI":"10.1007\/s11063-014-9379-0","article-title":"Classification of skeletal wireframe representation of hand gesture using complex-valued neural network","volume":"42","author":"Hafiz","year":"2015","journal-title":"Neural Process. Lett"},{"key":"B22","doi-asserted-by":"publisher","first-page":"2341","DOI":"10.1109\/TPAMI.2010.168","article-title":"Single image haze removal using dark channel prior","volume":"33","author":"He","year":"2010","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B23","doi-asserted-by":"publisher","first-page":"13890","DOI":"10.1021\/acs.chemrev.7b00218","article-title":"Super-resolution structured illumination microscopy","volume":"117","author":"Heintzmann","year":"2017","journal-title":"Chem. Rev"},{"key":"B24","doi-asserted-by":"publisher","first-page":"541","DOI":"10.1109\/TNNLS.2012.2183613","article-title":"Generalization characteristics of complex-valued feedforward neural networks in relation to signal coherence","volume":"23","author":"Hirose","year":"2012","journal-title":"IEEE Transact. Neural Netw. Learn. Syst"},{"key":"B25","first-page":"4634","article-title":"\u201cAttention on attention for image captioning,\u201d","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Huang","year":"2019"},{"key":"B26","doi-asserted-by":"publisher","first-page":"451","DOI":"10.1038\/nbt.4115","article-title":"Fast, long-term, super-resolution imaging with hessian structured illumination microscopy","volume":"36","author":"Huang","year":"2018","journal-title":"Nat. Biotechnol"},{"key":"B27","article-title":"\u201cLearn to pay attention,\u201d","volume-title":"International Conference on Learning Representations","author":"Jetley","year":"2018"},{"key":"B28","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41467-020-15784-x","article-title":"Deep learning enables structured illumination microscopy with low light levels and enhanced speed","volume":"11","author":"Jin","year":"2020","journal-title":"Nat. Commun"},{"key":"B29","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1145\/3065386","article-title":"Imagenet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"B30","doi-asserted-by":"publisher","first-page":"1406","DOI":"10.1109\/JAS.2022.105743","article-title":"Complex-valued neural networks: a comprehensive survey","volume":"9","author":"Lee","year":"2022","journal-title":"IEEE\/CAA J. Automat. Sinica"},{"key":"B31","doi-asserted-by":"crossref","first-page":"3824","DOI":"10.18653\/v1\/D18-1417","article-title":"\u201cA self-attentive model with gate mechanism for spoken language understanding,\u201d","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Li","year":"2018"},{"key":"B32","doi-asserted-by":"publisher","first-page":"102774","DOI":"10.1016\/j.jvcir.2020.102774","article-title":"Detail retaining convolutional neural network for image denoising","volume":"71","author":"Li","year":"2020","journal-title":"J. Vis. Commun. Image Represent."},{"key":"B33","first-page":"136","article-title":"\u201cEnhanced deep residual networks for single image super-resolution,\u201d","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops","author":"Lim","year":"2017"},{"key":"B34","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/JSTQE.2023.3279341","article-title":"Improving reconstruction of structured illumination microscopy images via dual-domain learning","volume":"29","author":"Liu","year":"2023","journal-title":"IEEE J. Select. Top. Quant. Electron"},{"key":"B35","doi-asserted-by":"publisher","first-page":"2810","DOI":"10.5555\/3157382.3157412","article-title":"Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections","volume":"29","author":"Mao","year":"2016","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B36","first-page":"416","article-title":"\u201cA database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics,\u201d","volume-title":"Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, Vol. 2","author":"Martin","year":"2001"},{"key":"B37","doi-asserted-by":"publisher","first-page":"2204","DOI":"10.5555\/2969033.2969073","article-title":"Recurrent models of visual attention","volume":"27","author":"Mnih","year":"2014","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B38","doi-asserted-by":"publisher","first-page":"980","DOI":"10.1038\/ncomms10980","article-title":"Open-source image reconstruction of super-resolution structured illumination microscopy data in imagej","volume":"7","author":"M\u00fcller","year":"2016","journal-title":"Nat. Commun"},{"key":"B39","first-page":"1099","article-title":"\u201cOn the critical points of the complex-valued neural network,\u201d","volume-title":"Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP'02, Vol. 3","author":"Nitta","year":"2002"},{"key":"B40","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1804.03999","article-title":"Attention U-Net: learning where to look for the pancreas","author":"Oktay","year":"2018","journal-title":"arXiv"},{"key":"B41","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1145\/3508072.3508081","article-title":"\u201cEfficient complex valued neural network with fourier transform on image denoising,\u201d","volume-title":"The 5th International Conference on Future Networks and Distributed Systems","author":"Pham","year":"2021"},{"key":"B42","first-page":"300","article-title":"\u201cFourier transform-based image classification using complex-valued convolutional neural networks,\u201d","volume-title":"International Symposium on Neural Networks","author":"Popa","year":"2018"},{"key":"B43","doi-asserted-by":"publisher","first-page":"194","DOI":"10.1038\/s41592-020-01048-5","article-title":"Evaluation and development of deep neural networks for image super-resolution in optical microscopy","volume":"18","author":"Qiao","year":"2021","journal-title":"Nat. Methods"},{"key":"B44","doi-asserted-by":"publisher","first-page":"107639","DOI":"10.1016\/j.patcog.2020.107639","article-title":"Image denoising using complex-valued deep cnn","volume":"111","author":"Quan","year":"2021","journal-title":"Pattern Recognit"},{"key":"B45","first-page":"5301","article-title":"\u201cOn the spectral bias of neural networks,\u201d","volume-title":"International Conference on Machine Learning","author":"Rahaman","year":"2019"},{"key":"B46","doi-asserted-by":"publisher","first-page":"102859","DOI":"10.1016\/j.bspc.2021.102859","article-title":"A novel complex-valued convolutional neural network for medical image denoising","volume":"69","author":"Rawat","year":"2021","journal-title":"Biomed. Signal Process. Control"},{"key":"B47","first-page":"234","article-title":"\u201cU-net: convolutional networks for biomedical image segmentation,\u201d","volume-title":"Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18","author":"Ronneberger","year":"2015"},{"key":"B48","doi-asserted-by":"publisher","first-page":"72","DOI":"10.1038\/s41556-018-0251-8","article-title":"Super-resolution microscopy demystified","volume":"21","author":"Schermelleh","year":"2019","journal-title":"Nat. Cell Biol"},{"key":"B49","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511815911","volume-title":"Statistical Signal Processing of Complex-Valued Data: The Theory of Improper and Noncircular Signals","author":"Schreier","year":"2010"},{"key":"B50","doi-asserted-by":"publisher","first-page":"plae038","DOI":"10.1093\/gigascience\/giad109","article-title":"Evaluation of swin transformer and knowledge transfer for denoising of super-resolution structured illumination microscopy data","volume":"16","author":"Shah","year":"2023","journal-title":"Gigascience"},{"key":"B51","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/IJCNN55064.2022.9892936","article-title":"\u201cImpact of different loss functions on denoising of microscopic images,\u201d","volume-title":"2022 International Joint Conference on Neural Networks (IJCNN)","author":"Shah","year":"2022"},{"key":"B52","doi-asserted-by":"publisher","DOI":"10.1364\/PRJ.416437","article-title":"Deep-learning based denoising and reconstruction of super-resolution structured illumination microscopy images","author":"Shah","year":"2021","journal-title":"Photon. Res"},{"key":"B53","doi-asserted-by":"publisher","first-page":"110306","DOI":"10.1016\/j.knosys.2023.110306","article-title":"Uncertainty-guided hierarchical frequency domain transformer for image restoration","volume":"263","author":"Shao","year":"2023","journal-title":"Knowl. Based Syst"},{"key":"B54","doi-asserted-by":"publisher","first-page":"821","DOI":"10.1038\/s41592-021-01167-7","article-title":"Structured illumination microscopy with noise-controlled image reconstructions","volume":"18","author":"Smith","year":"2021","journal-title":"Nat. Methods"},{"key":"B55","doi-asserted-by":"publisher","first-page":"667","DOI":"10.1364\/OPTICA.3.000667","article-title":"Frontiers in structured illumination microscopy","volume":"3","author":"Str\u00f6hl","year":"2016","journal-title":"Optica"},{"key":"B56","first-page":"1920","article-title":"\u201cAnchored neighborhood regression for fast example-based super-resolution,\u201d","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Timofte","year":"2013"},{"key":"B57","first-page":"111","article-title":"\u201cA+: adjusted anchored neighborhood regression for fast super-resolution,\u201d","volume-title":"Asian Conference on Computer Vision","author":"Timofte","year":"2014"},{"key":"B58","article-title":"\u201cGan-based noise model for denoising real images,\u201d","volume-title":"Proceedings of the Asian Conference on Computer Vision (ACCV)","author":"Tran","year":"2020"},{"key":"B59","doi-asserted-by":"publisher","first-page":"2257","DOI":"10.1587\/transinf.E96.D.2257","article-title":"An approach for sound source localization by complex-valued neural network","volume":"96","author":"Tsuzuki","year":"2013","journal-title":"IEICE Trans. Inf. Syst"},{"key":"B60","doi-asserted-by":"crossref","first-page":"3953","DOI":"10.1109\/ICIP.2017.8297024","article-title":"\u201cBetter than real: complex-valued neural nets for mri fingerprinting,\u201d","volume-title":"2017 IEEE International Conference on Image Processing (ICIP)","author":"Virtue","year":"2017"},{"key":"B61","first-page":"3156","article-title":"\u201cResidual attention network for image classification,\u201d","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Wang","year":"2017"},{"key":"B62","doi-asserted-by":"publisher","first-page":"341","DOI":"10.5555\/2999134.2999173","article-title":"Image denoising and inpainting with deep neural networks","volume":"25","author":"Xie","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B63","first-page":"1740","article-title":"\u201cLearning in the frequency domain,\u201d","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Xu","year":"2020"},{"key":"B64","first-page":"8280","article-title":"\u201cRecurrent saliency transformation network: incorporating multi-stage visual cues for small organ segmentation,\u201d","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Yu","year":"2018"},{"key":"B65","doi-asserted-by":"publisher","first-page":"4608","DOI":"10.1109\/TIP.2018.2839891","article-title":"Ffdnet: toward a fast and flexible solution for CNN-based image denoising","volume":"27","author":"Zhang","year":"2018","journal-title":"IEEE Transact. Image Process"},{"key":"B66","doi-asserted-by":"publisher","first-page":"043037","DOI":"10.1117\/1.JEI.27.4.043037","article-title":"S-net: a scalable convolutional neural network for jpeg compression artifact reduction","volume":"27","author":"Zheng","year":"2018","journal-title":"J. Electron. Imaging"},{"key":"B67","doi-asserted-by":"publisher","first-page":"020901","DOI":"10.1063\/5.0038065","article-title":"Current challenges and solutions of super-resolution structured illumination microscopy","volume":"6","author":"Zheng","year":"2021","journal-title":"APL Photon"},{"key":"B68","doi-asserted-by":"publisher","first-page":"6625688","DOI":"10.1155\/2021\/6625688","article-title":"R2AU-Net: attention recurrent residual convolutional neural network for multimodal medical image segmentation","volume":"2021","author":"Zuo","year":"2021","journal-title":"Sec. Commun. Netw"},{"key":"B69","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1007\/978-3-319-96029-6_4","article-title":"\u201cConvolutional neural networks for image denoising and restoration,\u201d","volume-title":"Denoising of Photographic Images and Video","author":"Zuo","year":"2018"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2024.1353873\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,23]],"date-time":"2024-09-23T08:35:08Z","timestamp":1727080508000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2024.1353873\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,23]]},"references-count":69,"alternative-id":["10.3389\/frai.2024.1353873"],"URL":"https:\/\/doi.org\/10.3389\/frai.2024.1353873","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,23]]},"article-number":"1353873"}}