{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T14:53:47Z","timestamp":1781880827619,"version":"3.54.5"},"reference-count":38,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2023,2,27]],"date-time":"2023-02-27T00:00:00Z","timestamp":1677456000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62072274"],"award-info":[{"award-number":["62072274"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Shandong Provincial Transfer and Transformation Project of Scientific and Technological Achievements","award":["2021LYXZ011"],"award-info":[{"award-number":["2021LYXZ011"]}]},{"name":"Shandong-Chongqing Science and Technology Cooperation Project","award":["cstc2021-lyjsAX0003"],"award-info":[{"award-number":["cstc2021-lyjsAX0003"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2023,7,31]]},"abstract":"<jats:p>Multi-modal medical image fusion is a long-standing important research topic that can obtain informative medical images and assist doctors diagnose and treat diseases more efficiently. However, most fusion methods extract and fuse features by subjectively defining constraints, which easily distorts the unique information of source images. In this work, we present a novel end-to-end unsupervised network to fuse multi-modal medical images. It is composed of a generator and two symmetrical discriminators. The former aims to generate a \u201dreal-like\u201d fused image based on a specifically designed content and structure loss, while the latter are devoted to distinguishing the differences between the fused image and the source ones. They are trained alternately until discriminators cannot distinguish the fused image from the source ones. In addition, the symmetrical discriminator scheme is conducive to maintaining the feature consistency among different modalities. More importantly, to enhance the retention degree of texture details, U-Net is adopted as the generator heuristically, where the up-sampling method is modified to bilinear interpolation for avoiding checkerboard artifacts. As for the optimization, we define the content loss function, which preserves the gradient information and pixel activity of source images. Both visual analysis and quantitative evaluation of experimental results show the superiority of our method as compared to the cutting-edge baselines.<\/jats:p>","DOI":"10.1145\/3574136","type":"journal-article","created":{"date-parts":[[2023,1,17]],"date-time":"2023-01-17T12:34:41Z","timestamp":1673958881000},"page":"1-17","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["DDIFN: A Dual-discriminator Multi-modal Medical Image Fusion Network"],"prefix":"10.1145","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1789-4616","authenticated-orcid":false,"given":"Hui","family":"Liu","sequence":"first","affiliation":[{"name":"School of Computer Science and Technology, Shandong University of Finance and Economics, Shandong Key Laboratory of Digital Media Technology, Jinan, Shandong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0477-9344","authenticated-orcid":false,"given":"Shanshan","family":"Li","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Shandong University of Finance and Economics, Shandong Key Laboratory of Digital Media Technology, Jinan, Shandong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5672-082X","authenticated-orcid":false,"given":"Jicheng","family":"Zhu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Shandong University of Finance and Economics, Shandong Key Laboratory of Digital Media Technology, Jinan, Shandong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4753-8564","authenticated-orcid":false,"given":"Kai","family":"Deng","sequence":"additional","affiliation":[{"name":"The First Affiliated Hospital of Shandong First Medical University, Jinan, Shandong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1582-5764","authenticated-orcid":false,"given":"Meng","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Shandong Jianzhu University, Jinan, Shandong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4753-8564","authenticated-orcid":false,"given":"Liqiang","family":"Nie","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, Shandong Key Laboratory of Digital Media Technology, Jinan, Shandong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,2,27]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"Martin Arjovsky Soumith Chintala and L\u00e9on Bottou. 2017. Wasserstain GAN. arXiv preprint arXiv:1701.07875. Retrieved from https:\/\/arxiv.org\/abs\/1701.07875."},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2013.2244870"},{"key":"e_1_3_2_4_2","first-page":"347","article-title":"Generative adversarial networks and their applications in image generation","volume":"44","author":"Chen Foji","year":"2021","unstructured":"Foji Chen, Feng Zhu, Qu Qingxiao, Hao Yinming, Ende Wan, and Yunge Cui. 2021. Generative adversarial networks and their applications in image generation. Chin. J. Comput. 44, 2 (2021), 347\u2013369.","journal-title":"Chin. J. Comput."},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/26.477498"},{"key":"e_1_3_2_6_2","unstructured":"Fanda Fan Yunyou Huang Lei Wang Xingwang Xiong Zihan Jiang Zhifei Zhang and Jianfeng Zhan. 2019. A semantic-based medical image fusion. arXiv preprint arXiv:1906.00225 . Retrieved from https:\/\/arxiv.org\/abs\/1906.00225."},{"key":"e_1_3_2_7_2","first-page":"2672","volume-title":"Proceedings of the International Conference on Advances in Neural Information Processing Systems.","author":"Goodfellow Ian","year":"2014","unstructured":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial networks. In Proceedings of the International Conference on Advances in Neural Information Processing Systems.2672\u20132680."},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2011.08.002"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.sigpro.2021.108036"},{"key":"e_1_3_2_10_2","doi-asserted-by":"crossref","unstructured":"Diclehan Karakaya Oguzhan Ulucan and Mehmet Turkan. 2021. PAS-MEF: Multi-exposure image fusion based on principal component analysis adaptive well-exposedness and saliency map. In Proceedings of the 2022 IEEE International Conference on Acoustics Speech and Signal Processing . 2345\u20132349.","DOI":"10.1109\/ICASSP43922.2022.9746779"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2015.03.003"},{"key":"e_1_3_2_12_2","first-page":"1","volume-title":"Proceedings of the Advances in Robotics.","author":"Kumar Aditya","year":"1979","unstructured":"Aditya Kumar, Atman Patel, and Santosha Dwivedy. 1979. Development of a NAO humanoid based medical assistant. In Proceedings of the Advances in Robotics.1\u20136."},{"key":"e_1_3_2_13_2","first-page":"1","volume-title":"Proceedings of the 22th International Conference on Information Fusion.","author":"Lahoud Fayez","year":"2019","unstructured":"Fayez Lahoud and Sabine S\u00fcsstrunk. 2019. Zero-learning fast medical image fusion. In Proceedings of the 22th International Conference on Information Fusion.1\u20138."},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIM.2020.2975405"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/LSP.2016.2618776"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2018.09.004"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2020.2977573"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infrared.2017.02.005"},{"key":"e_1_3_2_19_2","first-page":"2794","volume-title":"Proceedings of the IEEE International Conference on Computer Vision.","author":"Mao Xudong","year":"2017","unstructured":"Xudong Mao, Qing Li, Haoran Xie, Raymond Lau, Zhen Wang, and Stephen Paul Smolley. 2017. Least squares generative adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision.2794\u20132802."},{"key":"e_1_3_2_20_2","first-page":"386","volume-title":"Proceedings of the IEEE International Conference on Image Processing.","author":"Perra Cristian","year":"2005","unstructured":"Cristian Perra, Francesco Massidda, and Daniele D. Giusto. 2005. Image blockiness evaluation based on sobel operator. In Proceedings of the IEEE International Conference on Image Processing.386\u2013389."},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1049\/el:20020212"},{"key":"e_1_3_2_22_2","unstructured":"Alecv Radford Luke Metz and Soumith Chintala. 2016. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 . Retrieved from https:\/\/arxiv.org\/abs\/1511.06434."},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1088\/0957-0233\/8\/4\/002"},{"key":"e_1_3_2_24_2","first-page":"917","volume-title":"Proceedings of the 51st Annual Allerton Conference on Communication, Control, and Computing.","author":"Ratliff Lillian","year":"2013","unstructured":"Lillian Ratliff, Samuel Burden, and Shankar Sastry. 2013. Characterization and computation of local nash equilibria in continuous games. In Proceedings of the 51st Annual Allerton Conference on Communication, Control, and Computing.917\u2013924."},{"key":"e_1_3_2_25_2","first-page":"1","article-title":"Assessment of image fusion procedures using entropy, image quality, and multispectral classification","volume":"2","author":"Roberts Wesley","year":"2008","unstructured":"Wesley Roberts, Jan Andreas Van Aardt, and Fethi Babikker Ahmed. 2008. Assessment of image fusion procedures using entropy, image quality, and multispectral classification. J. Appl. Remote Sens. 2, 1 (2008), 1\u201328.","journal-title":"J. Appl. Remote Sens."},{"key":"e_1_3_2_26_2","first-page":"234","volume-title":"Proceedings of Medical Image Computing and Computer-Assisted Intervention.","author":"Ronneberger Olaf","year":"2015","unstructured":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional networks for biomedical image segmentation. In Proceedings of Medical Image Computing and Computer-Assisted Intervention.234\u2013241."},{"key":"e_1_3_2_27_2","doi-asserted-by":"crossref","unstructured":"Xiaohan Wang Linchao Zhu Yu Wu and Yi Yang. 2020. Symbiotic attention with privileged information for egocentric action recognition. In Proceedings of the AAAI Conference on Artificial Intelligence . 12249\u201312256.","DOI":"10.1609\/aaai.v34i07.6907"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3440248"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2003.819861"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1049\/el:20000267"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1631\/FITEE.2100463"},{"key":"e_1_3_2_32_2","first-page":"21","article-title":"Medical image fusion method by deep learning","volume":"2","author":"Yi Li","year":"2021","unstructured":"Li Yi, Zhao Junli, Lv Zhihan, and Li Jinhua. 2021. Medical image fusion method by deep learning. Int. J. Cogn. Comput. Eng. 2 (2021), 21\u201329.","journal-title":"Int. J. Cogn. Comput. Eng."},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIM.2018.2838778"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2022.108772"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2021.03.059"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2021.06.008"},{"key":"e_1_3_2_37_2","first-page":"12797","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence.","author":"Zhang Hao","year":"2019","unstructured":"Hao Zhang, Han Xu, Xiao Yang, Xiaojie Guo, and Jiayi Ma. 2019. Rethinking the image fusion: A fast unified image fusion network based on proportional maintenance of gradient and intensity. In Proceedings of the AAAI Conference on Artificial Intelligence.12797\u201312804."},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2019.07.011"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2015.11.003"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3574136","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3574136","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T18:08:25Z","timestamp":1750183705000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3574136"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,27]]},"references-count":38,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,7,31]]}},"alternative-id":["10.1145\/3574136"],"URL":"https:\/\/doi.org\/10.1145\/3574136","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,27]]},"assertion":[{"value":"2022-04-13","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-11-30","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-02-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}