{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T04:54:32Z","timestamp":1781844872095,"version":"3.54.5"},"reference-count":39,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T00:00:00Z","timestamp":1781827200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurol."],"abstract":"<jats:p>\n                    As a key technique in clinical diagnosis, multimodal medical image fusion (MMIF) integrates functional and metabolic information to assist diagnosis and enhance disease analysis reliability. However, existing methods typically rely on a single optimization objective, failing to meet clinical demands for flexible, on-the-fly result adjustments. To address this, we propose FreeMMIF, an interactive framework integrating a vision-language model (VLM)-based pseudo-labeling strategy and an instruction-aware diffusion model for task-guided, preference-adaptive fusion. During training, a VLM with specific prompts selects optimal candidates from existing methods as pseudo-ground truths to provide robust supervision without reference images. Subsequently, the diffusion process is modulated via an adaptive feature re-weighting branch, training the model to dynamically coordinate outputs by balancing weighted inputs from source images and pseudo-ground truths. Finally, we employ prompt engineering to construct a weight-generating VLM, allowing physicians to adjust source modality ratios via text instructions for direct control over fusion results. Experimental results demonstrate that FreeMMIF generates diagnostic-quality fusion results that precisely align with clinical intentions. Our code is available at:\n                    <jats:ext-link>https:\/\/github.com\/hzbbucm\/FreeMMIF<\/jats:ext-link>\n                    .\n                  <\/jats:p>","DOI":"10.3389\/fneur.2026.1802423","type":"journal-article","created":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T04:30:09Z","timestamp":1781843409000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["FreeMMIF: interactive multimodal medical image fusion via instruction-aware diffusion"],"prefix":"10.3389","volume":"17","author":[{"given":"Huizhong","family":"Bai","sequence":"first","affiliation":[{"name":"The First Clinical Medical School, Beijing University of Chinese Medicine","place":["Beijing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gang","family":"Liu","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, Tsinghua University","place":["Beijing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bowen","family":"Deng","sequence":"additional","affiliation":[{"name":"The First Clinical Medical School, Beijing University of Chinese Medicine","place":["Beijing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiashu","family":"Yue","sequence":"additional","affiliation":[{"name":"The First Clinical Medical School, Beijing University of Chinese Medicine","place":["Beijing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yu","family":"Jiang","sequence":"additional","affiliation":[{"name":"The First Clinical Medical School, Beijing University of Chinese Medicine","place":["Beijing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaoye","family":"Li","sequence":"additional","affiliation":[{"name":"The First Clinical Medical School, Beijing University of Chinese Medicine","place":["Beijing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jinyu","family":"Li","sequence":"additional","affiliation":[{"name":"The First Clinical Medical School, Beijing University of Chinese Medicine","place":["Beijing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaohong","family":"Mu","sequence":"additional","affiliation":[{"name":"The First Clinical Medical School, Beijing University of Chinese Medicine","place":["Beijing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,6,19]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"102666","DOI":"10.1016\/j.inffus.2024.102666","article-title":"MMIF-INet: multimodal medical image fusion by invertible network","volume":"114","author":"He","year":"2025","journal-title":"Inform Fusion"},{"key":"B2","doi-asserted-by":"publisher","first-page":"103687","DOI":"10.1016\/j.ipm.2024.103687","article-title":"FATFusion: A functional\u2013anatomical transformer for medical image fusion","volume":"61","author":"Tang","year":"2024","journal-title":"Inform Proc Managem"},{"key":"B3","first-page":"4733","article-title":"FreeU: free lunch in diffusion U-Net","volume-title":"IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Si","year":"2024"},{"key":"B4","first-page":"10684","article-title":"High-resolution image synthesis with latent diffusion models","volume-title":"IEEE\/CVF Conference on Computer Vision and Pattern Recognition.","author":"Rombach","year":"2022"},{"key":"B5","doi-asserted-by":"publisher","first-page":"2266","DOI":"10.3390\/electronics14112266","article-title":"Multimodal medical image fusion using a progressive parallel strategy based on deep learning","volume":"14","author":"Peng","year":"2025","journal-title":"Electronics"},{"key":"B6","doi-asserted-by":"publisher","first-page":"97","DOI":"10.3390\/electronics12010097","article-title":"A brief analysis of multimodal medical image fusion techniques","volume":"12","author":"Saleh","year":"2023","journal-title":"Electronics"},{"key":"B7","doi-asserted-by":"publisher","first-page":"35","DOI":"10.3233\/THC-2000-8104","article-title":"PACS and multimodality in medical imaging","volume":"8","author":"D'Asseler","year":"2000","journal-title":"Technol Health Care"},{"key":"B8","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1016\/B978-0-12-815553-0.00006-9","article-title":"Chapter 6 - A Medical Image Retrieval System in PACS Environment for Clinical Decision Making","author":"Jeyakumar","year":"2019","journal-title":"Intelligent Data Analysis for Biomedical Applications. Intelligent Data-Centric Systems"},{"key":"B9","doi-asserted-by":"publisher","first-page":"5134","DOI":"10.1109\/TIP.2022.3193288","article-title":"Multimodal medical image fusion via multiscale adaptive transformer","volume":"31","author":"Tang","year":"2022","journal-title":"IEEE Trans Image Proc"},{"key":"B10","doi-asserted-by":"publisher","first-page":"177","DOI":"10.1016\/j.inffus.2021.06.001","article-title":"EMFusion: An unsupervised enhanced medical image fusion network","volume":"76","author":"Xu","year":"2021","journal-title":"Inform Fusion"},{"key":"B11","doi-asserted-by":"publisher","first-page":"302","DOI":"10.1016\/j.ins.2021.04.052","article-title":"Multimodal medical image fusion based on joint bilateral filter and local gradient energy","volume":"569","author":"Li","year":"2021","journal-title":"Inf Sci"},{"key":"B12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TIM.2023.3317470","article-title":"MRSCFusion: joint residual swin transformer and multiscale CNN for unsupervised multimodal medical image fusion","volume":"72","author":"Xie","year":"2023","journal-title":"IEEE Trans Instrum Meas"},{"key":"B13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TIM.2022.3177717","article-title":"Infrared and visible image fusion based on adversarial feature extraction and stable image reconstruction","volume":"71","author":"Su","year":"2022","journal-title":"IEEE Trans Instrum Meas"},{"key":"B14","doi-asserted-by":"publisher","first-page":"3574136","DOI":"10.1145\/3574136","article-title":"DDIFN: a dual-discriminator multi-modal medical image fusion network","volume":"19","author":"Liu","year":"2023","journal-title":"ACM Trans Multimedia Comput Commun Appl"},{"key":"B15","first-page":"22490","article-title":"LayoutDiffusion: controllable diffusion model for layout-to-image generation","author":"Zheng","year":"2023","journal-title":"IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)"},{"key":"B16","first-page":"4429","article-title":"MatFuse: controllable material generation with diffusion models","author":"Vecchio","year":"2024","journal-title":"IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)"},{"key":"B17","first-page":"10167","article-title":"LayoutDM: discrete diffusion model for controllable layout generation","author":"Inoue","year":"2023","journal-title":"IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)"},{"key":"B18","doi-asserted-by":"publisher","first-page":"6935","DOI":"10.1609\/aaai.v38i7.28519","article-title":"Controllable mind visual diffusion model","volume":"38","author":"Zeng","year":"2024","journal-title":"AAAI Conf Artif Intellig"},{"key":"B19","first-page":"7677","article-title":"StyleDiffusion: controllable disentangled style transfer via diffusion models","author":"Wang","year":"2023","journal-title":"IEEE\/CVF International Conference on Computer Vision (ICCV)"},{"key":"B20","doi-asserted-by":"publisher","first-page":"5625","DOI":"10.1109\/TPAMI.2024.3369699","article-title":"Vision-language models for vision tasks: A survey","volume":"46","author":"Zhang","year":"2024","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"B21","first-page":"5693","article-title":"Deep stereo video inpainting","author":"Wu","year":"2023","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B22","first-page":"49250","article-title":"InstructBLIP: towards general-purpose vision-language models with instruction tuning","volume-title":"Advances in Neural Information Processing Systems","author":"Dai","year":"2023"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2412.19437","article-title":"Deepseek-v3 technical report","author":"Liu","year":"2024","journal-title":"arXiv"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2505.09388","article-title":"Qwen technical report","author":"Bai","year":"2023","journal-title":"arXiv"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.08774","article-title":"Gpt-4 technical report","author":"Achiam","year":"2023","journal-title":"arXiv"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2312.11805","article-title":"Gemini: a family of highly capable multimodal models","author":"Team","year":"2023","journal-title":"arXiv"},{"key":"B27","first-page":"2087","article-title":"LLaVA-CoT: let vision language models reason step-by-step","volume-title":"IEEE\/CVF International Conference on Computer Vision (ICCV).","author":"Xu","year":"2025"},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.5812\/ijcm.103160","article-title":"Computed tomography\/magnetic resonance imaging (CT\/MRI) image registration and fusion assessment for accurate glioblastoma radiotherapy treatment planning","author":"Mokhtar","year":"2020","journal-title":"Int J Cancer Managem"},{"key":"B29","doi-asserted-by":"publisher","first-page":"e321","DOI":"10.1055\/s-0040-1701603","article-title":"CT-MRI image fusion-based computer-assisted navigation management of communicative tumors involved the infratemporal-middle cranial fossa","volume":"82","author":"Yang","year":"2021","journal-title":"J Neurol Surg Part B: Skull Base"},{"key":"B30","doi-asserted-by":"publisher","first-page":"939","DOI":"10.5312\/wjo.v15.i10.939","article-title":"Impact of computed tomography\/magnetic resonance imaging registration on rehabilitation after percutaneous endoscopic decompression for lumbar stenosis: Retrospective study","volume":"15","author":"Guo","year":"2024","journal-title":"World J Orthop"},{"key":"B31","doi-asserted-by":"publisher","first-page":"108036","DOI":"10.1016\/j.sigpro.2021.108036","article-title":"Multimodal medical image fusion review: Theoretical background and recent advances","volume":"183","author":"Hermessi","year":"2021","journal-title":"Signal Processing"},{"key":"B32","doi-asserted-by":"publisher","first-page":"111768","DOI":"10.1016\/j.dib.2025.111768","article-title":"paired CT and MRI dataset for advanced medical imaging applications","volume":"61","author":"Siam","year":"2025","journal-title":"Data in Brief"},{"key":"B33","doi-asserted-by":"crossref","first-page":"8082","DOI":"10.1007\/978-3-030-96530-3","article-title":"DDFM: Denoising diffusion model for multi-modality image fusion","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision.","author":"Zhao","year":"2023"},{"key":"B34","first-page":"25912","article-title":"Equivariant multi-modality image fusion","author":"Zhao","year":"2024","journal-title":"IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B35","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1016\/j.inffus.2019.07.011","article-title":"A general image fusion framework based on convolutional neural network","volume":"54","author":"Zhang","year":"2020","journal-title":"Inform Fusion"},{"key":"B36","doi-asserted-by":"publisher","first-page":"634","DOI":"10.2174\/1573405616666201118123220","article-title":"Multi-modal medical image fusion algorithm based on spatial frequency motivated PA-PCNN in the NSST domain","volume":"17","author":"Vanitha","year":"2021","journal-title":"Curr Med Imag"},{"key":"B37","doi-asserted-by":"publisher","first-page":"7458","DOI":"10.1109\/JSEN.2019.2921803","article-title":"Coupled GAN with relativistic discriminators for infrared and visible images fusion","volume":"21","author":"Li","year":"2019","journal-title":"IEEE Sens J"},{"key":"B38","doi-asserted-by":"publisher","first-page":"502","DOI":"10.1109\/TPAMI.2020.3012548","article-title":"U2Fusion: a unified unsupervised image fusion network","volume":"44","author":"Xu","year":"2020","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"B39","first-page":"28102","article-title":"One Model for ALL: low-level task interaction is a key to task-agnostic image fusion","volume-title":"Computer Vision and Pattern Recognition Conference.","author":"Cheng","year":"2025"}],"container-title":["Frontiers in Neurology"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fneur.2026.1802423\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T04:30:13Z","timestamp":1781843413000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fneur.2026.1802423\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,19]]},"references-count":39,"alternative-id":["10.3389\/fneur.2026.1802423"],"URL":"https:\/\/doi.org\/10.3389\/fneur.2026.1802423","relation":{},"ISSN":["1664-2295"],"issn-type":[{"value":"1664-2295","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,19]]},"article-number":"1802423"}}