{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,3]],"date-time":"2026-10-03T03:04:28Z","timestamp":1790996668116,"version":"4.1.0"},"reference-count":45,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2023,10,14]],"date-time":"2023-10-14T00:00:00Z","timestamp":1697241600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JPM"],"abstract":"<jats:p>Multimodal neuroimaging has gained traction in Alzheimer\u2019s Disease (AD) diagnosis by integrating information from multiple imaging modalities to enhance classification accuracy. However, effectively handling heterogeneous data sources and overcoming the challenges posed by multiscale transform methods remains a significant hurdle. This article proposes a novel approach to address these challenges. To harness the power of diverse neuroimaging data, we employ a strategy that leverages optimized convolution techniques. These optimizations include varying kernel sizes and the incorporation of instance normalization, both of which play crucial roles in feature extraction from magnetic resonance imaging (MRI) and positron emission tomography (PET) images. Specifically, varying kernel sizes allow us to adapt the receptive field to different image characteristics, enhancing the model\u2019s ability to capture relevant information. Furthermore, we employ transposed convolution, which increases spatial resolution of feature maps, and it is optimized with varying kernel sizes and instance normalization. This heightened resolution facilitates the alignment and integration of data from disparate MRI and PET data. The use of larger kernels and strides in transposed convolution expands the receptive field, enabling the model to capture essential cross-modal relationships. Instance normalization, applied to each modality during the fusion process, mitigates potential biases stemming from differences in intensity, contrast, or scale between modalities. This enhancement contributes to improved model performance by reducing complexity and ensuring robust fusion. The performance of the proposed fusion method is assessed on three distinct neuroimaging datasets, which include: Alzheimer\u2019s Disease Neuroimaging Initiative (ADNI), consisting of 50 participants each at various stages of AD for both MRI and PET (Cognitive Normal, AD, and Early Mild Cognitive); Open Access Series of Imaging Studies (OASIS), consisting of 50 participants each at various stages of AD for both MRI and PET (Cognitive Normal, Mild Dementia, Very Mild Dementia); and whole-brain atlas neuroimaging (AANLIB) (consisting of 50 participants each at various stages of AD for both MRI and PET (Cognitive Normal, AD). To evaluate the quality of the fused images generated via our method, we employ a comprehensive set of evaluation metrics, including Structural Similarity Index Measurement (SSIM), which assesses the structural similarity between two images; Peak Signal-to-Noise Ratio (PSNR), which measures how closely the generated image resembles the ground truth; Entropy (E), which assesses the amount of information preserved or lost during fusion; the Feature Similarity Indexing Method (FSIM), which assesses the structural and feature similarities between two images; and Edge-Based Similarity (EBS), which measures the similarity of edges between the fused and ground truth images. The obtained fused image is further evaluated using a Mobile Vision Transformer. In the classification of AD vs. Cognitive Normal, the model achieved an accuracy of 99.00%, specificity of 99.00%, and sensitivity of 98.44% on the AANLIB dataset.<\/jats:p>","DOI":"10.3390\/jpm13101496","type":"journal-article","created":{"date-parts":[[2023,10,14]],"date-time":"2023-10-14T14:38:42Z","timestamp":1697294322000},"page":"1496","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":41,"title":["Optimized Convolutional Fusion for Multimodal Neuroimaging in Alzheimer\u2019s Disease Diagnosis: Enhancing Data Integration and Feature Extraction"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1082-185X","authenticated-orcid":false,"given":"Modupe","family":"Odusami","sequence":"first","affiliation":[{"name":"Department of Multimedia Engineering, Kaunas University of Technology, 51423 Kaunas, Lithuania"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2809-2213","authenticated-orcid":false,"given":"Rytis","family":"Maskeli\u016bnas","sequence":"additional","affiliation":[{"name":"Department of Multimedia Engineering, Kaunas University of Technology, 51423 Kaunas, Lithuania"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9990-1084","authenticated-orcid":false,"given":"Robertas","family":"Dama\u0161evi\u010dius","sequence":"additional","affiliation":[{"name":"Department of Applied Informatics, Vytautas Magnus University, 53361 Kaunas, Lithuania"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,10,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1589","DOI":"10.1001\/jama.2019.4782","article-title":"Diagnosis and Management of Dementia: A Review","volume":"322","author":"Arvanitakis","year":"2019","journal-title":"JAMA"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1186\/s13195-020-00612-7","article-title":"Imaging biomarkers in neurodegeneration: Current and future practices","volume":"12","author":"Young","year":"2020","journal-title":"Alzheimer\u2019s Res. Ther."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1186\/s13024-019-0325-5","article-title":"Neuroimaging Biomarkers for Alzheimer\u2019s Disease","volume":"14","author":"Yassa","year":"2019","journal-title":"Mol. Neurodegener."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"11258","DOI":"10.1038\/s41598-018-29295-9","article-title":"Structural brain imaging in Alzheimer\u2019s disease and mild cognitive impairment: Biomarker analysis and shared morphometry database","volume":"8","author":"Ledig","year":"2018","journal-title":"Sci. Rep."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"134971","DOI":"10.1016\/j.neulet.2020.134971","article-title":"Deep learning based mild cognitive impairment diagnosis using structure MR images","volume":"730","author":"Jiang","year":"2020","journal-title":"Neurosci. Lett."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"101929","DOI":"10.1016\/j.nicl.2019.101929","article-title":"Cortical graph neural network for AD and MCI diagnosis and transfer learning across populations","volume":"23","author":"Wee","year":"2019","journal-title":"NeuroImage Clin."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Yue, L., Gong, X., Chen, K., Mao, M., Li, J., Nandi, A.K., and Li, M. (2018, January 28\u201330). Auto-Detection of Alzheimer\u2019s Disease Using Deep Convolutional Neural Networks. Proceedings of the 2018 14th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD), Huangshan, China.","DOI":"10.1109\/FSKD.2018.8687207"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Gorji, H.T., and Kaabouch, N. (2019). A Deep Learning approach for Diagnosis of Mild Cognitive Impairment Based on MRI Images. Brain Sci., 9.","DOI":"10.3390\/brainsci9090217"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Kim, J., Jeong, M., Stiles, W.R., and Choi, H.S. (2022). Neuroimaging Modalities in Alzheimer\u2019s Disease: Diagnosis and Clinical Features. Int. J. Mol. Sci., 23.","DOI":"10.3390\/ijms23116079"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1007\/s40846-023-00801-3","article-title":"Explainable Deep-Learning-Based Diagnosis of Alzheimer\u2019s Disease Using Multimodal Input Fusion of PET and MRI Images","volume":"43","author":"Odusami","year":"2023","journal-title":"J. Med. Biol. Eng."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Ozsahin, I., Sekeroglu, B., and Mok, G.S.P. (2019). The use of back propagation neural networks and 18F-Florbetapir PET for early detection of Alzheimer\u2019s disease using Alzheimer\u2019s Disease Neuroimaging Initiative database. PLoS ONE, 14.","DOI":"10.1371\/journal.pone.0226577"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"659","DOI":"10.1016\/j.sjbs.2019.12.004","article-title":"The risk prediction of Alzheimer\u2019s disease based on the deep learning model of brain 18F-FDG positron emission tomography","volume":"27","author":"Yang","year":"2020","journal-title":"Saudi J. Biol. Sci."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.neuroimage.2014.05.018","article-title":"General overview on the merits of multimodal neuroimaging data fusion","volume":"102","author":"Roebroeck","year":"2014","journal-title":"NeuroImage"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1186\/s13244-022-01358-6","article-title":"Multimodality imaging of neurodegenerative disorders with a focus on multiparametric magnetic resonance and molecular imaging","volume":"14","author":"Loftus","year":"2023","journal-title":"Insights Into Imaging"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"911220","DOI":"10.3389\/fnagi.2022.911220","article-title":"Multi-Modal Neuroimaging Neural Network-Based Feature Detection for Diagnosis of Alzheimer\u2019s Disease","volume":"14","author":"Meng","year":"2022","journal-title":"Front. Aging Neurosci."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ismail, W.N., Rajeena, P.F., and Ali, M.A.S. (2022). MULTforAD: Multimodal MRI Neuroimaging for Alz-heimer\u2019s Disease Detection Based on a 3D Convolution Model. Electronics, 11.","DOI":"10.3390\/electronics11233893"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"512","DOI":"10.3390\/make5020031","article-title":"Alzheimer\u2019s Disease Detection from Fused PET and MRI Modalities Using an Ensemble Classifier","volume":"5","author":"Shukla","year":"2023","journal-title":"Mach. Learn. Knowl. Extr."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Das, A., and Bhattacharya, M. Effective image fusion method to study Alzheimer\u2019s disease using MR, PET images. Proceedings of the 2015 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Washington, DC, USA.","DOI":"10.1109\/BIBM.2015.7359915"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"101663","DOI":"10.1016\/j.compmedimag.2019.101663","article-title":"Hypergraph based multi-task feature selection for multimodal classification of Alzheimer\u2019s disease","volume":"80","author":"Shao","year":"2020","journal-title":"Comput. Med. Imaging Graph."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"101630","DOI":"10.1016\/j.media.2019.101630","article-title":"Multi-modal latent space inducing ensemble SVM classifier for early dementia diagnosis with neuroimaging data","volume":"60","author":"Zhou","year":"2020","journal-title":"Med. Image Anal."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"637386","DOI":"10.3389\/fdgth.2021.637386","article-title":"An Effective Multimodal Image Fusion Method Using MRI and PET for Alzheimer\u2019s Disease Diagnosis","volume":"3","author":"Song","year":"2021","journal-title":"Front. Digit. Health"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1109\/MMUL.2022.3156471","article-title":"Multimodal Fusion-Based Deep Learning Network for Effective Diagnosis of Alzheimer\u2019s Disease","volume":"29","author":"Dwivedi","year":"2022","journal-title":"IEEE Multimed."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Odusami, M., Maskeli\u016bnas, R., and Dama\u0161evi\u010dius, R. (2023). Pixel-Level Fusion Approach with Vision Transformer for Early Detection of Alzheimer\u2019s Disease. Electronics, 12.","DOI":"10.3390\/electronics12051218"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"012037","DOI":"10.1088\/1742-6596\/2173\/1\/012037","article-title":"Research on Multi-modal Medical Image Fusion Based on Multi-Scale Attention Mechanism","volume":"2173","author":"He","year":"2022","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Odusami, M., Maskeli\u016bnas, R., and Dama\u0161evi\u010dius, R. (2022). An Intelligent System for Early Recognition of Alzheimer\u2019s Disease Using Neuroimaging. Sensors, 22.","DOI":"10.3390\/s22030740"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3505244","article-title":"Transformers in vision: A survey","volume":"54","author":"Khan","year":"2022","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_27","unstructured":"Beyer, L., Zhai, X., and Kolesnikov, A. (2022). Better plain ViT baselines for ImageNet-1k. arXiv."},{"key":"ref_28","unstructured":"Zhou, D., Kang, B., Jin, X., Yang, L., Lian, X., Jiang, Z., Hou, Q., and Feng, J. (2021). Deepvit: Towards deeper vision transformer. arXiv."},{"key":"ref_29","unstructured":"Mehta, S., and Rastegari, M. (2021). Mobilevit: Light-weight, general-purpose, and mobile-friendly vision transformer. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"57094","DOI":"10.1109\/ACCESS.2022.3178607","article-title":"Kriging-Weighted Laplacian Kernels for Grayscale Image Sharpening","volume":"10","author":"Pham","year":"2022","journal-title":"IEEE Access"},{"key":"ref_31","unstructured":"Burduk, R., Kurzynski, M., and Wozniak, M. (2020). Advances in Intelligent Systems and Computing, Springer International Publishing."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"107733","DOI":"10.1016\/j.sigpro.2020.107733","article-title":"Laplacian feature detection and feature alignment for multimodal ophthalmic image registration using phase correlation and Hessian affine feature space","volume":"177","author":"Shan","year":"2020","journal-title":"Signal Process."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"168084","DOI":"10.1016\/j.ijleo.2021.168084","article-title":"An infrared and visible image fusion method based on VGG-19 network","volume":"248","author":"Zhou","year":"2021","journal-title":"Optik"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"103745","DOI":"10.1016\/j.dsp.2022.103745","article-title":"Infrared and visible image fusion based on tight frame learning via VGG19 network","volume":"131","author":"Lu","year":"2022","journal-title":"Digit. Signal Process."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"10574","DOI":"10.1007\/s10489-022-03745-4","article-title":"Transpose convolution based model for super-resolution image reconstruction","volume":"53","author":"Sahito","year":"2023","journal-title":"Appl. Intell."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Odusami, M., Maskeli\u016bnas, R., and Dama\u0161evi\u010dius, R. (2023). Pareto Optimized Adaptive Learning with Transposed Convolution for Image Fusion Alzheimer\u2019s Disease Classification. Brain Sci., 13.","DOI":"10.3390\/brainsci13071045"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Li, X., Wang, W., Hu, X., and Yang, J. (2019, January 15\u201320). Selective Kernel Networks. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA. Available online: https:\/\/openaccess.thecvf.com\/content_CVPR_2019\/html\/Li_Selective_Kernel_Networks_CVPR_2019_paper.html.","DOI":"10.1109\/CVPR.2019.00060"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"104859","DOI":"10.1016\/j.compag.2019.104859","article-title":"Crop yield prediction with deep convolutional neural networks","volume":"163","author":"Nevavuori","year":"2019","journal-title":"Comput. Electron. Agric."},{"key":"ref_39","unstructured":"Chi, L., Jiang, B., and Mu, Y. (2020). Advances in Neural Information Processing Systems, Curran Associates, Inc.. Available online: https:\/\/proceedings.neurips.cc\/paper\/2020\/hash\/2fd5d41ec6cfab47e32164d5624269b1-Abstract.html."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Zhu, Q., Bai, H., Sun, J., Cheng, C., and Li, X. (2022). LPAdaIN: Light Progressive Attention Adaptive Instance Normalization Model for Style Transfer. Electronics, 11.","DOI":"10.3390\/electronics11182929"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Chen, L., Lu, X., Zhang, J., Chu, X., and Chen, C. (2021, January 19\u201325). Hinet: Half instance normalization network for image restoration. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Virtual.","DOI":"10.1109\/CVPRW53098.2021.00027"},{"key":"ref_42","unstructured":"Ulyanov, D., Vedaldi, A., and Lempitsky, V. (2016). Instance normalization: The missing ingredient for fast stylization. arXiv."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"5016614","DOI":"10.1109\/TIM.2021.3109379","article-title":"Unifusion: A lightweight unified image fusion network","volume":"70","author":"Cheng","year":"2021","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_44","unstructured":"Singh, V. (2020, January 21). Image Fusion Metrics\u2014A Systematic Study. Proceedings of the 2nd International Conference on IoT, Social, Mobile, Analytics & Cloud in Computational Vision & Bio-Engineering (ISMAC-CVB 2020), Rochester, NY, USA."},{"key":"ref_45","first-page":"607","article-title":"Joint Learning of Image Deblurring and Depth Estimation Through Adversarial Multitask Network","volume":"9","author":"Hou","year":"2023","journal-title":"IEEE Trans. Circuits Syst. Video Technol."}],"container-title":["Journal of Personalized Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4426\/13\/10\/1496\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:07:00Z","timestamp":1760130420000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4426\/13\/10\/1496"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,14]]},"references-count":45,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2023,10]]}},"alternative-id":["jpm13101496"],"URL":"https:\/\/doi.org\/10.3390\/jpm13101496","relation":{},"ISSN":["2075-4426"],"issn-type":[{"value":"2075-4426","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,14]]}}}