{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,18]],"date-time":"2026-01-18T09:14:13Z","timestamp":1768727653341,"version":"3.49.0"},"reference-count":55,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2022,3,23]],"date-time":"2022-03-23T00:00:00Z","timestamp":1647993600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004543","name":"China Scholarship Council","doi-asserted-by":"publisher","award":["201806220060"],"award-info":[{"award-number":["201806220060"]}],"id":[{"id":"10.13039\/501100004543","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In this paper, we propose a unified and flexible framework for general image fusion tasks, including multi-exposure image fusion, multi-focus image fusion, infrared\/visible image fusion, and multi-modality medical image fusion. Unlike other deep learning-based image fusion methods applied to a fixed number of input sources (normally two inputs), the proposed framework can simultaneously handle an arbitrary number of inputs. Specifically, we use the symmetrical function (e.g., Max-pooling) to extract the most significant features from all the input images, which are then fused with the respective features from each input source. This symmetry function enables permutation-invariance of the network, which means the network can successfully extract and fuse the saliency features of each image without needing to remember the input order of the inputs. The property of permutation-invariance also brings convenience for the network during inference with unfixed inputs. To handle multiple image fusion tasks with one unified framework, we adopt continual learning based on Elastic Weight Consolidation (EWC) for different fusion tasks. Subjective and objective experiments on several public datasets demonstrate that the proposed method outperforms state-of-the-art methods on multiple image fusion tasks.<\/jats:p>","DOI":"10.3390\/s22072457","type":"journal-article","created":{"date-parts":[[2022,3,23]],"date-time":"2022-03-23T22:08:06Z","timestamp":1648073286000},"page":"2457","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["General Image Fusion for an Arbitrary Number of Inputs Using Convolutional Neural Networks"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9040-6394","authenticated-orcid":false,"given":"Yifan","family":"Xiao","sequence":"first","affiliation":[{"name":"Department of Telecommunications and Information Processing, IPI-IMEC, Ghent University, 9000 Ghent, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhixin","family":"Guo","sequence":"additional","affiliation":[{"name":"Department of Telecommunications and Information Processing, IPI-IMEC, Ghent University, 9000 Ghent, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4746-9087","authenticated-orcid":false,"given":"Peter","family":"Veelaert","sequence":"additional","affiliation":[{"name":"Department of Telecommunications and Information Processing, IPI-IMEC, Ghent University, 9000 Ghent, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wilfried","family":"Philips","sequence":"additional","affiliation":[{"name":"Department of Telecommunications and Information Processing, IPI-IMEC, Ghent University, 9000 Ghent, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,3,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1016\/j.inffus.2016.05.004","article-title":"Pixel-level image fusion: A survey of the state of the art","volume":"33","author":"Li","year":"2017","journal-title":"Inf. Fusion"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.inffus.2019.07.005","article-title":"Infrared and visible image fusion via detail preserving adversarial learning","volume":"54","author":"Ma","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1016\/j.inffus.2020.06.013","article-title":"Multi-focus image fusion: A Survey of the state of the art","volume":"64","author":"Liu","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1016\/j.inffus.2017.10.007","article-title":"Deep learning for pixel-level image fusion: Recent advances and future prospects","volume":"42","author":"Liu","year":"2018","journal-title":"Inf. Fusion"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2614","DOI":"10.1109\/TIP.2018.2887342","article-title":"DenseFuse: A fusion approach to infrared and visible images","volume":"28","author":"Li","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"4816","DOI":"10.1109\/TIP.2020.2976190","article-title":"DRPL: Deep Regression Pair Learning for Multi-Focus Image Fusion","volume":"29","author":"Li","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Ram Prabhakar, K., Sai Srikar, V., and Venkatesh Babu, R. (2017, January 22\u201329). Deepfuse: A deep unsupervised approach for exposure fusion with extreme exposure image pairs. Proceedings of the IEEE International Conference on Computer Vision 2017, Venice, Italy.","DOI":"10.1109\/ICCV.2017.505"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"502","DOI":"10.1109\/TPAMI.2020.3012548","article-title":"U2Fusion: A unified unsupervised image fusion network","volume":"44","author":"Xu","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Kou, L., Zhang, L., Zhang, K., Sun, J., Han, Q., and Jin, Z. (2018). A multi-focus image fusion method via region mosaicking on Laplacian pyramids. PLoS ONE, 13.","DOI":"10.1371\/journal.pone.0191085"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1855","DOI":"10.1016\/j.patcog.2004.03.010","article-title":"A wavelet-based image fusion tutorial","volume":"37","author":"Pajares","year":"2004","journal-title":"Pattern Recognit."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"196","DOI":"10.1016\/j.inffus.2011.01.002","article-title":"The multiscale directional bilateral filter and its application to multisensor image fusion","volume":"13","author":"Hu","year":"2012","journal-title":"Inf. Fusion"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/j.acha.2007.09.003","article-title":"Sparse directional image representations using the discrete shearlet transform","volume":"25","author":"Easley","year":"2008","journal-title":"Appl. Comput. Harmon. Anal."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"057006","DOI":"10.1117\/1.OE.52.5.057006","article-title":"Dictionary learning method for joint sparse representation-based image fusion","volume":"52","author":"Zhang","year":"2013","journal-title":"Opt. Eng."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1049\/iet-ipr.2014.0311","article-title":"Simultaneous image fusion and denoising with adaptive sparse representation","volume":"9","author":"Liu","year":"2015","journal-title":"IET Image Process."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1193","DOI":"10.1007\/s11760-013-0556-9","article-title":"Image fusion based on pixel significance using cross bilateral filter","volume":"9","author":"Kumar","year":"2015","journal-title":"Signal Image Video Process."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.dsp.2016.08.004","article-title":"Multi-focal image fusion using degree of focus and fuzzy logic","volume":"60","author":"Rahman","year":"2017","journal-title":"Digit. Signal Process."},{"key":"ref_17","first-page":"3583","article-title":"Block-based pixel level multi-focus image fusion using particle swarm optimization","volume":"7","author":"Siddiqui","year":"2011","journal-title":"Int. Innov. Comput. Inf. Control"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1948","DOI":"10.1016\/j.patrec.2006.05.004","article-title":"A region-based multi-sensor image fusion scheme using pulse-coupled neural network","volume":"27","author":"Li","year":"2006","journal-title":"Pattern Recognit. Lett."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1016\/j.inffus.2014.09.004","article-title":"A general framework for image fusion based on multi-scale transform and sparse representation","volume":"24","author":"Liu","year":"2015","journal-title":"Inf. Fusion"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Li, H., Wu, X.J., and Kittler, J. (2018, January 20\u201324). Infrared and visible image fusion using a deep learning framework. Proceedings of the 2018 24th International Conference on Pattern Recognition (ICPR), Beijing, China.","DOI":"10.1109\/ICPR.2018.8546006"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.inffus.2018.09.004","article-title":"FusionGAN: A generative adversarial network for infrared and visible image fusion","volume":"48","author":"Ma","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Xu, H., Liang, P., Yu, W., Jiang, J., and Ma, J. (2019, January 10\u201316). Learning a Generative Model for Fusing Infrared and Visible Images via Conditional Generative Adversarial Network with Dual Discriminators. Proceedings of the 2019 International Joint Conference on Artificial Intelligence, Macao, China.","DOI":"10.24963\/ijcai.2019\/549"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"7458","DOI":"10.1109\/JSEN.2019.2921803","article-title":"Coupled GAN with relativistic discriminators for infrared and visible images fusion","volume":"21","author":"Li","year":"2019","journal-title":"IEEE Sens. J."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1016\/j.inffus.2020.08.022","article-title":"MFF-GAN: An unsupervised generative adversarial network with adaptive and gradient joint constraints for multi-focus image fusion","volume":"66","author":"Zhang","year":"2021","journal-title":"Inf. Fusion"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1016\/j.ins.2017.12.043","article-title":"Pixel convolutional neural network for multi-focus image fusion","volume":"433","author":"Tang","year":"2018","journal-title":"Inf. Sci."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1016\/j.inffus.2016.12.001","article-title":"Multi-focus image fusion with a deep convolutional neural network","volume":"36","author":"Liu","year":"2017","journal-title":"Inf. Fusion"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1016\/j.inffus.2019.02.003","article-title":"Ensemble of CNN for multi-focus image fusion","volume":"51","author":"Aghagolzadeh","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"5793","DOI":"10.1007\/s00521-020-05358-9","article-title":"Sesf-fuse: An unsupervised deep model for multi-focus image fusion","volume":"33","author":"Ma","year":"2020","journal-title":"Neural Comput. Appl."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"9755","DOI":"10.1109\/JSEN.2019.2928818","article-title":"Multi-focus image fusion using u-shaped networks with a hybrid objective","volume":"19","author":"Li","year":"2019","journal-title":"IEEE Sens. J."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"3345","DOI":"10.1109\/TIP.2015.2442920","article-title":"Perceptual quality assessment for multi-exposure image fusion","volume":"24","author":"Ma","year":"2015","journal-title":"IEEE Trans. Image Process."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2808","DOI":"10.1109\/TIP.2019.2952716","article-title":"Deep guided learning for fast multi-exposure image fusion","volume":"29","author":"Ma","year":"2019","journal-title":"IEEE Trans. Image Process."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1016\/j.inffus.2019.07.011","article-title":"IFCNN: A general image fusion framework based on convolutional neural network","volume":"54","author":"Zhang","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Xu, H., Ma, J., Le, Z., Jiang, J., and Guo, X. (2020, January 7\u201312). Fusiondn: A unified densely connected network for image fusion. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i07.6936"},{"key":"ref_34","unstructured":"Lee, S.W., Kim, J.H., Jun, J., Ha, J.W., and Zhang, B.T. (2017). Overcoming catastrophic forgetting by incremental moment matching. arXiv."},{"key":"ref_35","unstructured":"Zaheer, M., Kottur, S., Ravanbakhsh, S., Poczos, B., Salakhutdinov, R., and Smola, A. (2017). Deep sets. arXiv."},{"key":"ref_36","unstructured":"Qi, C.R., Su, H., Mo, K., and Guibas, L.J. (2017, January 21\u201326). Pointnet: Deep learning on point sets for 3d classification and segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA."},{"key":"ref_37","first-page":"7211","article-title":"Mapping images to scene graphs with permutation-invariant structured prediction","volume":"31","author":"Herzig","year":"2018","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Aittala, M., and Durand, F. (2018, January 8\u201314). Burst image deblurring using permutation invariant convolutional neural networks. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01237-3_45"},{"key":"ref_39","unstructured":"Wagstaff, E., Fuchs, F., Engelcke, M., Posner, I., and Osborne, M.A. (2019, January 9\u201315). On the limitations of representing functions on sets. Proceedings of the International Conference on Machine Learning (PMLR), Vancouver, BC, Canada."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"3521","DOI":"10.1073\/pnas.1611835114","article-title":"Overcoming catastrophic forgetting in neural networks","volume":"114","author":"Kirkpatrick","year":"2017","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Agustsson, E., and Timofte, R. (2017, January 21\u201326). Ntire 2017 challenge on single image super-resolution: Dataset and study. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.","DOI":"10.1109\/CVPRW.2017.150"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"2049","DOI":"10.1109\/TIP.2018.2794218","article-title":"Learning a deep single image contrast enhancer from multi-exposure images","volume":"27","author":"Cai","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Prabhakar, K.R., and Babu, R.V. (2016, January 20\u201325). Ghosting-free multi-exposure image fusion in gradient domain. Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China.","DOI":"10.1109\/ICASSP.2016.7471980"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"2864","DOI":"10.1109\/TIP.2013.2244222","article-title":"Image fusion with guided filtering","volume":"22","author":"Li","year":"2013","journal-title":"IEEE Trans. Image Process."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Zhang, H., Xu, H., Xiao, Y., Guo, X., and Ma, J. (2020, January 7\u201312). Rethinking the image fusion: A fast unified image fusion network based on proportional maintenance of gradient and intensity. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i07.6975"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1890","DOI":"10.1016\/j.aeue.2015.09.004","article-title":"A new image quality metric for image fusion: The sum of the correlations of differences","volume":"69","author":"Aslantas","year":"2015","journal-title":"Aeu-Int. J. Electron. Commun."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1016\/j.inffus.2011.08.002","article-title":"A new image fusion performance metric based on visual information fidelity","volume":"14","author":"Han","year":"2013","journal-title":"Inf. Fusion"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Haghighat, M., and Razian, M.A. (2014, January 15\u201317). Fast-FMI: Non-reference image fusion metric. Proceedings of the 2014 IEEE 8th International Conference on Application of Information and Communication Technologies (AICT), Paris, France.","DOI":"10.1109\/ICAICT.2014.7036000"},{"key":"ref_50","unstructured":"Wang, P.W., and Liu, B. (2008, January 26\u201329). A novel image fusion metric based on multi-scale analysis. Proceedings of the 2008 9th International Conference on Signal Processing, Beijing, China."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1016\/j.inffus.2006.09.001","article-title":"A novel similarity based quality metric for image fusion","volume":"9","author":"Yang","year":"2008","journal-title":"Inf. Fusion"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"308","DOI":"10.1049\/el:20000267","article-title":"Objective image fusion performance measure","volume":"36","author":"Xydeas","year":"2000","journal-title":"Electron. Lett."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"5805","DOI":"10.1109\/TIP.2020.2987133","article-title":"Fast multi-scale structural patch decomposition for multi-exposure image fusion","volume":"29","author":"Li","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"1671","DOI":"10.1109\/LSP.2017.2752233","article-title":"Evaluating multiexposure fusion using image information","volume":"24","author":"Rahman","year":"2017","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1016\/j.inffus.2014.10.004","article-title":"Multi-focus image fusion using dictionary-based sparse representation","volume":"25","author":"Nejati","year":"2015","journal-title":"Inf. Fusion"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/7\/2457\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:41:25Z","timestamp":1760136085000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/7\/2457"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,23]]},"references-count":55,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2022,4]]}},"alternative-id":["s22072457"],"URL":"https:\/\/doi.org\/10.3390\/s22072457","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,23]]}}}