{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T02:37:38Z","timestamp":1780972658785,"version":"3.54.1"},"reference-count":68,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2024,12,25]],"date-time":"2024-12-25T00:00:00Z","timestamp":1735084800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key R&D Program of China","doi-asserted-by":"crossref","award":["2023YFE0116300"],"award-info":[{"award-number":["2023YFE0116300"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62006097, U1836218, 62202205"],"award-info":[{"award-number":["62006097, U1836218, 62202205"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100004608","name":"Natural Science Foundation of Jiangsu Province","doi-asserted-by":"crossref","award":["BK20200593"],"award-info":[{"award-number":["BK20200593"]}],"id":[{"id":"10.13039\/501100004608","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100002858","name":"China Postdoctoral Science Foundation","doi-asserted-by":"crossref","award":["2021M701456"],"award-info":[{"award-number":["2021M701456"]}],"id":[{"id":"10.13039\/501100002858","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"crossref","award":["JUSRP121074, JUSRP123030"],"award-info":[{"award-number":["JUSRP121074, JUSRP123030"]}],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2025,2,28]]},"abstract":"<jats:p>\n            Image restoration (IR) involves the retrieval of missing or damaged image information and represents a significant challenge in the field of visual reconstruction. Currently, U-Net based Diffusion Models (DMs) display favorable results when utilized for IR tasks. However, the DM based on U-Net demonstrates shortcomings in capturing the global context for IR. To address this issue, we propose a Novel Image Restoration Approach Based on U-shaped Transformer and DMs (DIRformer). DIRformer enhances the modeling capacity for long-range dependencies within DMs. In particular, DIRformer replaces the traditional U-Net downsampling with Patch merging, dedicated to improving detail preservation, and replaces upsampling with Dual up-sample, strategically designed to alleviate checkerboard artifacts. Besides, as a lightweight and versatile transformer-based solution for IR, DIRformer incorporates time and degradation mapping into the transformer design, all while preserving the fundamental U-shaped structural framework. We assess the efficacy of DIRformer in a multi-tasking IR setting across four datasets. The experimental performance illustrates that DIRformer achieves competitive performance on distortion metrics, including PSNR and SSIM. Remarkably, our proposed approach is almost 25\n            <jats:inline-formula content-type=\"math\/tex\">\n              <jats:tex-math notation=\"LaTeX\" version=\"MathJax\">\\(\\times\\)<\/jats:tex-math>\n            <\/jats:inline-formula>\n            smaller and 2\n            <jats:inline-formula content-type=\"math\/tex\">\n              <jats:tex-math notation=\"LaTeX\" version=\"MathJax\">\\(\\times\\)<\/jats:tex-math>\n            <\/jats:inline-formula>\n            faster than the existing methods while achieving comparable high performance.\n          <\/jats:p>","DOI":"10.1145\/3703632","type":"journal-article","created":{"date-parts":[[2024,11,8]],"date-time":"2024-11-08T15:23:22Z","timestamp":1731079402000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["DIRformer: A Novel Image Restoration Approach Based on U-shaped Transformer and Diffusion Models"],"prefix":"10.1145","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8473-077X","authenticated-orcid":false,"given":"Cong","family":"Hu","sequence":"first","affiliation":[{"name":"School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-2229-9237","authenticated-orcid":false,"given":"Xiao-Zhong","family":"Wei","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0310-5778","authenticated-orcid":false,"given":"Xiao-Jun","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,12,25]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"crossref","first-page":"3684","DOI":"10.1109\/TASLP.2024.3428908","article-title":"ASiT: Local-global audio spectrogram vision transformer for event Classification","volume":"32","author":"Atito Sara","year":"2024","unstructured":"Sara Atito, Muhammad Awais, Wenwu Wang, Mark D. Plumbley, and Josef Kittler. 2024. ASiT: Local-global audio spectrogram vision transformer for event Classification. IEEE ACM Trans. Audio Speech Lang. Process 32 (2024), 3684\u20133693.","journal-title":"IEEE ACM Trans. Audio Speech Lang. Process"},{"key":"e_1_3_1_3_2","first-page":"21757","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021 (NeurIPS \u201921)","author":"Kawar Gregory Vaksman Bahjat","year":"2021","unstructured":"Gregory Vaksman Bahjat Kawar and Michael Elad. 2021. SNIPS: Solving noisy inverse problems stochastically. In Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021 (NeurIPS \u201921). Marc\u2019Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (Eds.), 21757\u201321769."},{"key":"e_1_3_1_4_2","first-page":"22669","article-title":"All are worth words: A ViT backbone for score-based Diffusion Models","author":"Bao Fan","year":"2023","unstructured":"Fan Bao, Chongxuan Li, Yue Cao, Chongxuan Li, Hang Su and Jun Zhu. 2023. All are worth words: A ViT backbone for score-based Diffusion Models. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition CVPR 2023, 22669\u201322679.","journal-title":"IEEE\/CVF Conference on Computer Vision and Pattern Recognition CVPR 2023"},{"key":"e_1_3_1_5_2","doi-asserted-by":"crossref","first-page":"752","DOI":"10.1109\/IJCNN.2017.7965927","volume-title":"Proceedings of the International Joint Conference on Neural Networks (IJCNN \u201917)","author":"Cavigelli Lukas","year":"2017","unstructured":"Lukas Cavigelli, Pascal Hager, and Luca Benini. 2017. CAS-CNN: A deep convolutional neural network for image compression artifact suppression. In Proceedings of the International Joint Conference on Neural Networks (IJCNN \u201917). IEEE, 752\u2013759."},{"key":"e_1_3_1_6_2","first-page":"12299","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR \u201921)","author":"Chen Hanting","year":"2021","unstructured":"Hanting Chen, Yunhe Wang, Tianyu Guo, Chang Xu, Yiping Deng, Zhenhua Liu, Siwei Ma, Chunjing Xu, Chao Xu, and Wen Gao. 2021. Pre-trained image processing transformer. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR \u201921). Computer Vision Foundation\/IEEE, 12299\u201312310."},{"key":"e_1_3_1_7_2","series-title":"Lecture Notes in Computer Science","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1007\/978-3-031-20071-7_2","volume-title":"Proceedings of the 17th European Conference on Computer Vision (ECCV \u201922)","volume":"13667","author":"Chen Liangyu","year":"2022","unstructured":"Liangyu Chen, Xiaojie Chu, Xiangyu Zhang, and Jian Sun. 2022. Simple baselines for image restoration. In Proceedings of the 17th European Conference on Computer Vision (ECCV \u201922). Shai Avidan, Gabriel J. Brostow, Moustapha Ciss\u00e9, Giovanni Maria Farinella, and Tal Hassner (Eds.), Lecture Notes in Computer Science, Vol. 13667, Springer, 17\u201333."},{"key":"e_1_3_1_8_2","doi-asserted-by":"crossref","first-page":"9002","DOI":"10.1109\/TIP.2020.3023580","article-title":"Robust face super-resolution via position relation model based on global face context","volume":"29","author":"Chen Liang","year":"2020","unstructured":"Liang Chen, Jinshan Pan, Junjun Jiang, Jiawei Zhang, and Yi Wu. 2020. Robust face super-resolution via position relation model based on global face context. IEEE Trans. Image Process. 29 (2020), 9002\u20139016.","journal-title":"IEEE Trans. Image Process"},{"key":"e_1_3_1_9_2","first-page":"14347","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV \u201921)","author":"Choi Jooyoung","year":"2021","unstructured":"Jooyoung Choi, Sungwon Kim, Yonghyun Jeong, Youngjune Gwon, and Sungroh Yoon. 2021. ILVR: Conditioning method for denoising diffusion probabilistic models. In Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV \u201921). IEEE, 14347\u201314356."},{"key":"e_1_3_1_10_2","volume-title":"Proceedings of the 11th International Conference on Learning Representations (ICLR \u201923)","author":"Chung Hyungjin","year":"2023","unstructured":"Hyungjin Chung, Jeongsol Kim, Michael Thompson McCann, Marc Louis Klasky, and Jong Chul Ye. 2023. Diffusion posterior sampling for general noisy inverse problems. In Proceedings of the 11th International Conference on Learning Representations (ICLR \u201923). OpenReview.net."},{"key":"e_1_3_1_11_2","first-page":"12403","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201922)","author":"Chung Hyungjin","year":"2022","unstructured":"Hyungjin Chung, Byeongsu Sim, and Jong Chul Ye. 2022. Come-closer-diffuse-faster: Accelerating conditional diffusion models for inverse problems through stochastic contraction. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201922). IEEE, 12403\u201312412."},{"key":"e_1_3_1_12_2","first-page":"248","volume-title":"Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR \u201909)","author":"Deng Jia","year":"2009","unstructured":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR \u201909). IEEE Computer Society, 248\u2013255."},{"key":"e_1_3_1_13_2","first-page":"8780","volume-title":"Advances in Neural Information Processing Systems","author":"Dhariwal Prafulla","year":"2021","unstructured":"Prafulla Dhariwal and Alexander Quinn Nichol. 2021. Diffusion models beat GANs on image synthesis. In Advances in Neural Information Processing Systems. Marc\u2019Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (Eds.), Curran Associates, Inc., 8780\u20138794. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2021\/file\/49ad23d1ec9fa4bd8d77d02681df5cfa-Paper.pdf"},{"key":"e_1_3_1_14_2","first-page":"576","volume-title":"Proceedings of the IEEE International Conference on Computer Vision (ICCV \u201915)","author":"Dong Chao","year":"2015","unstructured":"Chao Dong, Yubin Deng, Chen Change Loy, and Xiaoou Tang. 2015. Compression artifacts reduction by a deep convolutional network. In Proceedings of the IEEE International Conference on Computer Vision (ICCV \u201915). IEEE Computer Society, 576\u2013584."},{"key":"e_1_3_1_15_2","volume-title":"Proceedings of the 9th International Conference on Learning Representations (ICLR \u201921)","author":"Dosovitskiy Alexey","year":"2021","unstructured":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An image is worth 16x16 words: Transformers for image recognition at scale. In Proceedings of the 9th International Conference on Learning Representations (ICLR \u201921). OpenReview.net."},{"key":"e_1_3_1_16_2","first-page":"12873","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR \u201921)","author":"Esser Patrick","year":"2021","unstructured":"Patrick Esser, Robin Rombach, and Bj\u00f6rn Ommer. 2021. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR \u201921). Computer Vision Foundation\/IEEE, 12873\u201312883."},{"key":"e_1_3_1_17_2","first-page":"9935","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201923)IEEE","author":"Fei Ben","year":"2023","unstructured":"Ben Fei, Zhaoyang Lyu, Liang Pan, Junzhe Zhang, Weidong Yang, Tianyue Luo, Bo Zhang, and Bo Dai. 2023. Generative diffusion prior for unified image restoration and enhancement. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201923). IEEE, 9935\u20139946."},{"key":"e_1_3_1_18_2","doi-asserted-by":"crossref","first-page":"1978","DOI":"10.1109\/TIP.2023.3261747","article-title":"CTCNet: A CNN-transformer cooperation network for face image super-resolution","volume":"32","author":"Gao Guangwei","year":"2023","unstructured":"Guangwei Gao, Zixiang Xu, Juncheng Li, Jian Yang, Tieyong Zeng, and Guo-Jun Qi. 2023. CTCNet: A CNN-transformer cooperation network for face image super-resolution. IEEE Trans. Image Process. 32 (2023), 1978\u20131991.","journal-title":"IEEE Trans. Image Process"},{"key":"e_1_3_1_19_2","doi-asserted-by":"crossref","first-page":"107539","DOI":"10.1016\/j.patcog.2020.107539","article-title":"Constructing multilayer locality-constrained matrix regression framework for noise robust face super-resolution","volume":"110","author":"Gao Guangwei","year":"2021","unstructured":"Guangwei Gao, Yi Yu, Jin Xie, Jian Yang, Meng Yang, and Jian Zhang. 2021. Constructing multilayer locality-constrained matrix regression framework for noise robust face super-resolution. Pattern Recognit. 110 (2021), 107539.","journal-title":"Pattern Recognit"},{"key":"e_1_3_1_20_2","first-page":"10686","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201922)IEEE","author":"Gu Shuyang","year":"2022","unstructured":"Shuyang Gu, Dong Chen, Jianmin Bao, Fang Wen, Bo Zhang, Dongdong Chen, Lu Yuan, and Baining Guo. 2022. Vector quantized diffusion model for text-to-image synthesis. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201922). IEEE, 10686\u201310696."},{"key":"e_1_3_1_21_2","first-page":"5767","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems.","author":"Gulrajani Ishaan","year":"2017","unstructured":"Ishaan Gulrajani, Faruk Ahmed, Mart\u00edn Arjovsky, Vincent Dumoulin, and Aaron C. Courville. 2017. Improved training of Wasserstein GANs. In Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems. Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.), 5767\u20135777."},{"key":"e_1_3_1_22_2","first-page":"4709","volume-title":"Proceedings of the 38th International Conference on Machine Learning (ICML \u201921)","volume":"139","author":"Jalal Ajil","year":"2021","unstructured":"Ajil Jalal, Sushrut Karmalkar, Alex Dimakis, and Eric Price. 2021. Instance-optimal compressed sensing via posterior sampling. In Proceedings of the 38th International Conference on Machine Learning (ICML \u201921), Vol. 139. Marina Meila and Tong Zhang (Eds.), PMLR, 4709\u20134720."},{"issue":"1","key":"e_1_3_1_23_2","first-page":"1","article-title":"Cascading blend network for image inpainting","volume":"20","author":"Jin Yiting","year":"2023","unstructured":"Yiting Jin, Jie Wu, Wanliang Wang, Yidong Yan, Jiawei Jiang, and Jianwei Zheng. 2023. Cascading blend network for image inpainting. ACM Trans. Multimedia Comput. Commun. Appl. 20, 1 (2023), 1\u201321.","journal-title":"ACM Trans. Multimedia Comput. Commun. Appl"},{"key":"e_1_3_1_24_2","first-page":"13242","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021 (NeurIPS \u201921)","author":"Kadkhodaie Zahra","year":"2021","unstructured":"Zahra Kadkhodaie and Eero P. Simoncelli. 2021. Stochastic solutions for linear inverse problems using the prior implicit in a denoiser. In Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021 (NeurIPS \u201921). Marc\u2019Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (Eds.), 13242\u201313254."},{"key":"e_1_3_1_25_2","volume-title":"Proceedings of the Advances in Neural Information Processing Systems (NeurIPS \u201922)","author":"Kawar Bahjat","year":"2022","unstructured":"Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. 2022. Denoising diffusion restoration models. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS \u201922)."},{"key":"e_1_3_1_26_2","first-page":"1866","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision Workshops (ICCVW \u201921)","author":"Kawar Bahjat","year":"2021","unstructured":"Bahjat Kawar, Gregory Vaksman, and Michael Elad. 2021. Stochastic image denoising by sampling from the posterior distribution. In Proceedings of the IEEE\/CVF International Conference on Computer Vision Workshops (ICCVW \u201921). IEEE, 1866\u20131875."},{"key":"e_1_3_1_27_2","first-page":"8877","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV \u201919)","author":"Kupyn Orest","year":"2019","unstructured":"Orest Kupyn, Tetiana Martyniuk, Junru Wu, and Zhangyang Wang. 2019. DeblurGAN-v2: Deblurring (orders-of-magnitude) faster and better. In Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV \u201919). IEEE, 8877\u20138886."},{"key":"e_1_3_1_28_2","first-page":"5800","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201920)Computer Vision Foundation\/IEEE","author":"Lee Junsoo","year":"2020","unstructured":"Junsoo Lee, Eungyeup Kim, Yunsung Lee, Dongjun Kim, Jaehyuk Chang, and Jaegul Choo. 2020. Reference-based sketch image colorization using augmented-self reference and dense semantic correspondence. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201920). Computer Vision Foundation\/IEEE, 5800\u20135809."},{"key":"e_1_3_1_29_2","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1016\/j.neucom.2022.01.029","article-title":"SRDiff: Single image super-resolution with diffusion probabilistic models","volume":"479","author":"Li Haoying","year":"2022","unstructured":"Haoying Li, Yifan Yang, Meng Chang, Shiqi Chen, Huajun Feng, Zhihai Xu, Qi Li, and Yueting Chen. 2022. SRDiff: Single image super-resolution with diffusion probabilistic models. Neurocomputing 479 (2022), 47\u201359.","journal-title":"Neurocomputing"},{"key":"e_1_3_1_30_2","first-page":"1833","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision Workshops (ICCVW \u201921)","author":"Liang Jingyun","year":"2021","unstructured":"Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, and Radu Timofte. 2021. SwinIR: Image restoration using swin transformer. In Proceedings of the IEEE\/CVF International Conference on Computer Vision Workshops (ICCVW \u201921). IEEE, 1833\u20131844."},{"issue":"1","key":"e_1_3_1_31_2","first-page":"1","article-title":"Sequential hierarchical learning with distribution transformation for image super-resolution","volume":"19","author":"Liu Yuqing","year":"2023","unstructured":"Yuqing Liu, Xinfeng Zhang, Shanshe Wang, Siwei Ma, and Wen Gao. 2023. Sequential hierarchical learning with distribution transformation for image super-resolution. ACM Trans. Multimedia Comput. Commun. Appl. 19, 1s (2023), 1\u201321.","journal-title":"ACM Trans. Multimedia Comput. Commun. Appl"},{"key":"e_1_3_1_32_2","first-page":"9992","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV \u201921)","author":"Liu Ze","year":"2021","unstructured":"Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV \u201921). IEEE, 9992\u201310002."},{"key":"e_1_3_1_33_2","first-page":"2041","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPR \u201919) Workshops","author":"Liu Zhi-Song","year":"2019","unstructured":"Zhi-Song Liu, Li-Wen Wang, Chu-Tak Li, and Wan-Chi Siu. 2019. Hierarchical back projection network for image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPR \u201919) Workshops. Computer Vision Foundation\/IEEE, 2041\u20132050."},{"key":"e_1_3_1_34_2","series-title":"Lecture Notes in Computer Science","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1007\/978-3-030-57321-8_15","volume-title":"Proceedings of the 4th IFIP TC 5, TC 12, WG 8.4, WG 8.9, WG 12.9 International Cross-Domain Conference on Machine Learning and Knowledge Extraction (CD-MAKE \u201920)","volume":"12279","author":"Lyn Jiawen","year":"2020","unstructured":"Jiawen Lyn and Sen Yan. 2020. Non-local second-order attention network for single image super resolution. In Proceedings of the 4th IFIP TC 5, TC 12, WG 8.4, WG 8.9, WG 12.9 International Cross-Domain Conference on Machine Learning and Knowledge Extraction (CD-MAKE \u201920). Andreas Holzinger, Peter Kieseberg, A. Min Tjoa, and Edgar R. Weippl (Eds.), Lecture Notes in Computer Science, Vol. 12279, Springer, 267\u2013279."},{"issue":"9","key":"e_1_3_1_35_2","doi-asserted-by":"crossref","first-page":"5830","DOI":"10.1109\/TCSVT.2022.3163649","article-title":"Designing CNNs for multimodal image restoration and fusion via unfolding the method of multipliers","volume":"32","author":"Marivani Iman","year":"2022","unstructured":"Iman Marivani, Evaggelia Tsiligianni, Bruno Cornelis, and Nikos Deligiannis. 2022. Designing CNNs for multimodal image restoration and fusion via unfolding the method of multipliers. IEEE Trans. Circuits Syst. Video Technol. 32, 9 (2022), 5830\u20135845.","journal-title":"IEEE Trans. Circuits Syst. Video Technol"},{"key":"e_1_3_1_36_2","first-page":"2434","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201920)","author":"Menon Sachit","year":"2020","unstructured":"Sachit Menon, Alexandru Damian, Shijia Hu, Nikhil Ravi, and Cynthia Rudin. 2020. PULSE: Self-supervised photo upsampling via latent space exploration of generative models. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201920). Computer Vision Foundation\/IEEE, 2434\u20132442."},{"issue":"11","key":"e_1_3_1_37_2","doi-asserted-by":"crossref","first-page":"7474","DOI":"10.1109\/TPAMI.2021.3115428","article-title":"Exploiting deep generative prior for versatile image restoration and manipulation","volume":"44","author":"Pan Xingang","year":"2022","unstructured":"Xingang Pan, Xiaohang Zhan, Bo Dai, Dahua Lin, Chen Change Loy, and Ping Luo. 2022. Exploiting deep generative prior for versatile image restoration and manipulation. IEEE Trans. Pattern Anal. Mach. Intell. 44, 11 (2022), 7474\u20137489.","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"e_1_3_1_38_2","first-page":"4172","volume-title":"IEEE\/CVF International Conference on Computer Vision, ICCV 2023","author":"Peebles William","year":"2023","unstructured":"William Peebles and Saining Xie. 2023. Scalable diffusion models with transformers. In IEEE\/CVF International Conference on Computer Vision, ICCV 2023. IEEE, 4172\u20134182."},{"key":"e_1_3_1_39_2","first-page":"1","article-title":"TGFuse: An infrared and visible image fusion approach based on transformer and generative adversarial network","author":"Rao Dongyu","year":"2023","unstructured":"Dongyu Rao, Tianyang Xu, and Xiao-Jun Wu. 2023. TGFuse: An infrared and visible image fusion approach based on transformer and generative adversarial network. IEEE Trans. Image Process. (2023), 1.","journal-title":"IEEE Trans. Image Process"},{"issue":"4","key":"e_1_3_1_40_2","doi-asserted-by":"crossref","first-page":"1804","DOI":"10.1137\/16M1102884","article-title":"The little engine that could: Regularization by denoising (RED)","volume":"10","author":"Romano Yaniv","year":"2017","unstructured":"Yaniv Romano, Michael Elad, and Peyman Milanfar. 2017. The little engine that could: Regularization by denoising (RED). SIAM J. Imaging Sci. 10, 4 (2017), 1804\u20131844.","journal-title":"SIAM J. Imaging Sci"},{"key":"e_1_3_1_41_2","first-page":"15:1","volume-title":"Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference (SIGGRAPH \u201922)","author":"Saharia Chitwan","year":"2022","unstructured":"Chitwan Saharia, William Chan, Huiwen Chang, Chris A. Lee, Jonathan Ho, Tim Salimans, David J. Fleet, and Mohammad Norouzi. 2022. Palette: Image-to-image diffusion models. In Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference (SIGGRAPH \u201922). Munkhtsetseg Nandigjav, Niloy J. Mitra, and Aaron Hertzmann (Eds.), ACM, 15:1\u201315:10."},{"issue":"4","key":"e_1_3_1_42_2","first-page":"4713","article-title":"Image Super-Resolution via Iterative Refinement","volume":"45","author":"Saharia Chitwan","year":"2023","unstructured":"Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J. Fleet, and Mohammad Norouzi. 2023. Image Super-Resolution via Iterative Refinement. IEEE Trans. Pattern Anal. Mach. Intell. 45, 4 (2023), 4713\u20134726.","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"e_1_3_1_43_2","first-page":"1874","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Shi Wenzhe","year":"2016","unstructured":"Wenzhe Shi, Jose Caballero, Ferenc Husz\u00e1r, Johannes Totz, Andrew P. Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. 2016. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1874\u20131883."},{"key":"e_1_3_1_44_2","doi-asserted-by":"crossref","first-page":"375","DOI":"10.1109\/TIP.2019.2928136","article-title":"Image compressed sensing using convolutional neural network","volume":"29","author":"Shi Wuzhen","year":"2020","unstructured":"Wuzhen Shi, Feng Jiang, Shaohui Liu, and Debin Zhao. 2020. Image compressed sensing using convolutional neural network. IEEE Trans. Image Process. 29 (2020), 375\u2013388.","journal-title":"IEEE Trans. Image Process"},{"key":"e_1_3_1_45_2","volume-title":"Proceedings of the 10th International Conference on Learning Representations (ICLR \u201922)","author":"Song Yang","year":"2022","unstructured":"Yang Song, Liyue Shen, Lei Xing, and Stefano Ermon. 2022. Solving inverse problems in medical imaging with score-based generative models. In Proceedings of the 10th International Conference on Learning Representations (ICLR \u201922). OpenReview.net."},{"key":"e_1_3_1_46_2","first-page":"3603","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201920)","author":"Suin Maitreya","year":"2020","unstructured":"Maitreya Suin, Kuldeep Purohit, and A. N. Rajagopalan. 2020. Spatially-attentive patch-hierarchical network for adaptive motion deblurring. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201920). Computer Vision Foundation\/IEEE, 3603\u20133612."},{"key":"e_1_3_1_47_2","doi-asserted-by":"crossref","first-page":"9482","DOI":"10.1109\/TIP.2020.3023629","article-title":"Dual-path attention network for compressed sensing image reconstruction","volume":"29","author":"Sun Yubao","year":"2020","unstructured":"Yubao Sun, Jiwei Chen, Qingshan Liu, Bo Liu, and Guodong Guo. 2020. Dual-path attention network for compressed sensing image reconstruction. IEEE Trans. Image Process. 29 (2020), 9482\u20139495.","journal-title":"IEEE Trans. Image Process"},{"key":"e_1_3_1_48_2","doi-asserted-by":"crossref","first-page":"1489","DOI":"10.1109\/TMM.2020.2999182","article-title":"Coarse-to-fine CNN for image super-resolution","volume":"23","author":"Tian Chunwei","year":"2021","unstructured":"Chunwei Tian, Yong Xu, Wangmeng Zuo, Bob Zhang, Lunke Fei, and Chia-Wen Lin. 2021. Coarse-to-fine CNN for image super-resolution. IEEE Trans. Multim. 23 (2021), 1489\u20131502.","journal-title":"IEEE Trans. Multim"},{"key":"e_1_3_1_49_2","first-page":"1","volume-title":"Proceedings of the 26th Signal Processing and Communications Applications Conference (SIU \u201918)","author":"Turhan Ceren Guzel","year":"2018","unstructured":"Ceren Guzel Turhan and Hasan Sakir Bilge. 2018. Single image super resolution using deep convolutional generative neural networks. In Proceedings of the 26th Signal Processing and Communications Applications Conference (SIU \u201918). IEEE, 1\u20134."},{"key":"e_1_3_1_50_2","first-page":"5998","volume-title":"Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017. Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.), 5998\u20136008."},{"key":"e_1_3_1_51_2","series-title":"Lecture Notes in Computer Science","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1007\/978-3-030-11021-5_5","volume-title":"Proceedings of the Computer Vision (ECCV \u201918) Workshops","volume":"11133","author":"Wang Xintao","year":"2018","unstructured":"Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Yu Qiao, and Chen Change Loy. 2018. ESRGAN: Enhanced super-resolution generative adversarial networks. In Proceedings of the Computer Vision (ECCV \u201918) Workshops. Laura Leal-Taix\u00e9 and Stefan Roth (Eds.), Lecture Notes in Computer Science, Vol. 11133, Springer, 63\u201379."},{"key":"e_1_3_1_52_2","unstructured":"Yinhuai Wang Jiwen Yu and Jian Zhang. 2022. Zero-shot image restoration using denoising diffusion null-space model. arXiv:2212.00490."},{"issue":"4","key":"e_1_3_1_53_2","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang Zhou","year":"2004","unstructured":"Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. 2004. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 13, 4 (2004), 600\u2013612.","journal-title":"IEEE Trans. Image Process"},{"key":"e_1_3_1_54_2","first-page":"17662","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201922)","author":"Wang Zhendong","year":"2022","unstructured":"Zhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang Zhou, Jianzhuang Liu, and Houqiang Li. 2022. Uformer: A general U-shaped transformer for image restoration. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201922). IEEE, 17662\u201317672."},{"key":"e_1_3_1_55_2","first-page":"16272","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201922)","author":"Whang Jay","year":"2022","unstructured":"Jay Whang, Mauricio Delbracio, Hossein Talebi, Chitwan Saharia, Alexandros G. Dimakis, and Peyman Milanfar. 2022. Deblurring via stochastic refinement. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201922). IEEE, 16272\u201316282."},{"key":"e_1_3_1_56_2","first-page":"2759","volume-title":"Proceedings of the 36th AAAI Conference on Artificial Intelligence, AAAI 2022, 34th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201922), Proceedings of the 12th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201922)","author":"Xia Bin","year":"2022","unstructured":"Bin Xia, Yucheng Hang, Yapeng Tian, Wenming Yang, Qingmin Liao, and Jie Zhou. 2022. Efficient non-local contrastive attention for image super-resolution. In Proceedings of the 36th AAAI Conference on Artificial Intelligence, AAAI 2022, 34th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201922), Proceedings of the 12th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201922). AAAI Press, 2759\u20132767."},{"key":"e_1_3_1_57_2","first-page":"8857","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV \u201919)","author":"Xie Chaohao","year":"2019","unstructured":"Chaohao Xie, Shaohui Liu, Chao Li, Ming-Ming Cheng, Wangmeng Zuo, Xiao Liu, Shilei Wen, and Errui Ding. 2019. Image inpainting with learnable bidirectional attention maps. In Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV \u201919). IEEE, 8857\u20138866."},{"issue":"4","key":"e_1_3_1_58_2","doi-asserted-by":"crossref","first-page":"1308","DOI":"10.1109\/TCSVT.2020.3001267","article-title":"E2I: Generative inpainting from edge to image","volume":"31","author":"Xu Shunxin","year":"2021","unstructured":"Shunxin Xu, Dong Liu, and Zhiwei Xiong. 2021. E2I: Generative inpainting from edge to image. IEEE Trans. Circuits Syst. Video Technol. 31, 4 (2021), 1308\u20131322.","journal-title":"IEEE Trans. Circuits Syst. Video Technol"},{"issue":"9","key":"e_1_3_1_59_2","doi-asserted-by":"crossref","first-page":"5736","DOI":"10.1109\/TCSVT.2022.3153685","article-title":"Unsupervised image restoration with quality-task-perception loss","volume":"32","author":"Xu Wei","year":"2022","unstructured":"Wei Xu, Xinyuan Chen, Haoming Guo, Xiaolin Huang, and Wei Liu. 2022. Unsupervised image restoration with quality-task-perception loss. IEEE Trans. Circuits Syst. Video Technol. 32, 9 (2022), 5736\u20135747.","journal-title":"IEEE Trans. Circuits Syst. Video Technol"},{"key":"e_1_3_1_60_2","unstructured":"Xiulong Yang Sheng-Min Shih Yinlin Fu Xiaoting Zhao and Shihao Ji. 2022. Your ViT is secretly a hybrid discriminative-generative diffusion model. arXiv:abs\/2208.07791."},{"key":"e_1_3_1_61_2","first-page":"7505","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201920)","author":"Yi Zili","year":"2020","unstructured":"Zili Yi, Qiang Tang, Shekoofeh Azizi, Daesik Jang, and Zhan Xu. 2020. Contextual residual aggregation for ultra high-resolution image inpainting. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201920). Computer Vision Foundation\/IEEE, 7505\u20137514."},{"key":"e_1_3_1_62_2","first-page":"5505","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR \u201918)","author":"Yu Jiahui","year":"2018","unstructured":"Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S. Huang. 2018. Generative image inpainting with contextual attention. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR \u201918). Computer Vision Foundation\/IEEE Computer Society, 5505\u20135514."},{"key":"e_1_3_1_63_2","first-page":"4470","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV \u201919)","author":"Yu Jiahui","year":"2019","unstructured":"Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S. Huang. 2019. Free-form image inpainting with gated convolution. In Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV \u201919). IEEE, 4470\u20134479."},{"key":"e_1_3_1_64_2","first-page":"5718","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201922)IEEE","author":"Zamir Syed Waqas","year":"2022","unstructured":"Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. 2022. Restormer: Efficient transformer for high-resolution image restoration. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201922). IEEE, 5718\u20135729."},{"issue":"10","key":"e_1_3_1_65_2","doi-asserted-by":"crossref","first-page":"6360","DOI":"10.1109\/TPAMI.2021.3088914","article-title":"Plug-and-play image restoration with deep denoiser prior","volume":"44","author":"Zhang Kai","year":"2022","unstructured":"Kai Zhang, Yawei Li, Wangmeng Zuo, Lei Zhang, Luc Van Gool, and Radu Timofte. 2022. Plug-and-play image restoration with deep denoiser prior. IEEE Trans. Pattern Anal. Mach. Intell. 44, 10 (2022), 6360\u20136376.","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"issue":"7","key":"e_1_3_1_66_2","doi-asserted-by":"crossref","first-page":"3142","DOI":"10.1109\/TIP.2017.2662206","article-title":"Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising","volume":"26","author":"Zhang Kai","year":"2017","unstructured":"Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. 2017. Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Trans. Image Process. 26, 7 (2017), 3142\u20133155.","journal-title":"IEEE Trans. Image Process"},{"issue":"1","key":"e_1_3_1_67_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3532625","article-title":"Image quality assessment\u2013driven reinforcement learning for mixed distorted image restoration","volume":"19","author":"Zhang Xiaoyu","year":"2023","unstructured":"Xiaoyu Zhang, Wei Gao, Ge Li, Qiuping Jiang, and Runmin Cong. 2023. Image quality assessment\u2013driven reinforcement learning for mixed distorted image restoration. ACM Trans. Multimedia Comput. Commun. Appl. 19, 1s (2023), 1\u201323.","journal-title":"ACM Trans. Multimedia Comput. Commun. Appl"},{"key":"e_1_3_1_68_2","series-title":"Lecture Notes in Computer Science","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1007\/978-3-030-01234-2_18","volume-title":"Proceedings of the 15th European Conference on Computer Vision (ECCV \u201918)","volume":"11211","author":"Zhang Yulun","year":"2018","unstructured":"Yulun Zhang, Kunpeng Li, Kai Li, Lichen Wang, Bineng Zhong, and Yun Fu. 2018. Image super-resolution using very deep residual channel attention networks. In Proceedings of the 15th European Conference on Computer Vision (ECCV \u201918). Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss (Eds.), Lecture Notes in Computer Science, Vol. 11211, Springer, 294\u2013310."},{"issue":"8","key":"e_1_3_1_69_2","doi-asserted-by":"crossref","first-page":"3062","DOI":"10.1109\/TCSVT.2020.3037688","article-title":"SCGAN: Saliency map-guided colorization with generative adversarial network","volume":"31","author":"Zhao Yuzhi","year":"2021","unstructured":"Yuzhi Zhao, Lai-Man Po, Kwok-Wai Cheung, Wing Yin Yu, and Yasar Abbas Ur Rehman. 2021. SCGAN: Saliency map-guided colorization with generative adversarial network. IEEE Trans. Circuits Syst. Video Technol. 31, 8 (2021), 3062\u20133077.","journal-title":"IEEE Trans. Circuits Syst. Video Technol"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3703632","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3703632","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:09:42Z","timestamp":1750295382000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3703632"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,25]]},"references-count":68,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,2,28]]}},"alternative-id":["10.1145\/3703632"],"URL":"https:\/\/doi.org\/10.1145\/3703632","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,25]]},"assertion":[{"value":"2024-01-07","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-11-03","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-12-25","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}