{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,25]],"date-time":"2026-05-25T07:03:35Z","timestamp":1779692615649,"version":"3.53.1"},"reference-count":31,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,5,25]],"date-time":"2026-05-25T00:00:00Z","timestamp":1779667200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100016221","name":"Universitatea Politehnica din Bucure\u0219i","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100016221","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:sec>\n                    <jats:title>Introduction<\/jats:title>\n                    <jats:p>Multi-modal image registration leverages complementary information from diverse imaging sources to achieve precise spatial alignment. However, aligning visible (VIS), near-infrared (NIR), and thermal (TH) modalities remains challenging due to appearance differences and limited annotated datasets.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>This study proposes a ResU-Net-inspired framework combining heatmap prediction and homography estimation to enable joint NIR-TH registration via shared feature representations. A proprietary dataset of 155 VIS-NIR-TH skin lesion triplets was created from 62 patients, resulting in 465 images in total. Diagnoses were confirmed by dermatologists or histopathology. Lesions were distributed across the face, trunk, and limbs to evaluate registration robustness under spatial variability.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results and discussion<\/jats:title>\n                    <jats:p>Experiments showed that NIR-VIS registration consistently outperformed TH-VIS registration, reflecting NIR's richer structural content and higher spectral similarity with VIS. Despite limitations related to dataset size and acquisition variability, the framework demonstrated the feasibility of VIS-NIR-TH triplet registration and provides the first documented dataset of its kind for multi-modal skin lesion imaging.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/frai.2026.1807482","type":"journal-article","created":{"date-parts":[[2026,5,25]],"date-time":"2026-05-25T06:15:45Z","timestamp":1779689745000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["A heatmap-based deep learning framework for multi-modal registration of VIS, NIR, and thermal images in dermatological imaging"],"prefix":"10.3389","volume":"9","author":[{"given":"Maria","family":"Oniga","sequence":"first","affiliation":[{"name":"Department of Applied Electronics and Information Engineering, National University of Science and Technology POLITEHNICA Bucharest","place":["Bucharest, Romania"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paul Florin","family":"Rus","sequence":"additional","affiliation":[{"name":"Department of Applied Electronics and Information Engineering, National University of Science and Technology POLITEHNICA Bucharest","place":["Bucharest, Romania"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rvazvan","family":"Condorovici","sequence":"additional","affiliation":[{"name":"Department of Applied Electronics and Information Engineering, National University of Science and Technology POLITEHNICA Bucharest","place":["Bucharest, Romania"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alina Elena","family":"Sultana","sequence":"additional","affiliation":[{"name":"Department of Applied Electronics and Information Engineering, National University of Science and Technology POLITEHNICA Bucharest","place":["Bucharest, Romania"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrei","family":"Marin","sequence":"additional","affiliation":[{"name":"Faculty of Medicine, \u201cCarol Davila\u201d University of Medicine and Pharmacy","place":["Bucharest, Romania"]},{"name":"Department of Plastic Surgery, St John's Clinical Emergency Hospital","place":["Bucharest, Romania"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cristina","family":"Cotru\u0163\u01ce","sequence":"additional","affiliation":[{"name":"Faculty of Medicine, \u201cCarol Davila\u201d University of Medicine and Pharmacy","place":["Bucharest, Romania"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel Octavian","family":"Costache","sequence":"additional","affiliation":[{"name":"2nd Dermatology Discipline, Faculty of Medicine, \u201cCarol Davila\u201d University of Medicine and Pharmacy","place":["Bucharest, Romania"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Horia","family":"Blejan","sequence":"additional","affiliation":[{"name":"Faculty of Medicine, \u201cCarol Davila\u201d University of Medicine and Pharmacy","place":["Bucharest, Romania"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Radu Cristian","family":"Jecan","sequence":"additional","affiliation":[{"name":"Department of Plastic Surgery and Reconstructive Microsurgery, \u201cCarol Davila\u201d University of Medicine and Pharmacy","place":["Bucharest, Romania"]},{"name":"Department of Plastic and Reconstructive Microsurgery, Emergency Clinical Hospital","place":["Bucharest, Romania"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Costinela","family":"Corciu","sequence":"additional","affiliation":[{"name":"Faculty of Medicine, \u201cCarol Davila\u201d University of Medicine and Pharmacy","place":["Bucharest, Romania"]},{"name":"Clinical Department of Plastic Surgery and Reconstructive Microsurgery, \u201cProf. Dr. Agrippa Ionescu\u201d Hospital","place":["Bucharest, Romania"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,5,25]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"729","DOI":"10.1109\/ICEE59167.2023.10334862","author":"Abdollahinejad","year":"2023"},{"key":"B2","first-page":"4264","article-title":"\u201cEarthmatch: Iterative coregistration for fine-grained localization of astronaut photography,\u201d","author":"Berton","year":"2024","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B3","doi-asserted-by":"publisher","first-page":"102615","DOI":"10.1016\/j.media.2022.102615","article-title":"Transmorph: Transformer for unsupervised medical image registration","volume":"82","author":"Chen","year":"2022","journal-title":"Med. Image Anal"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.23919\/FUSION49751.2022.9841256","article-title":"\u201cThermal and visible image registration using deep homography,\u201d","author":"Debaque","year":"2022","journal-title":"Proceedings of the 25th International Conference on Information Fusion (FUSION)"},{"key":"B5","doi-asserted-by":"publisher","first-page":"1078","DOI":"10.1109\/TIP.2023.3240024","article-title":"Interpretable multi-modal image registration network based on disentangled convolutional sparse coding","volume":"32","author":"Deng","year":"2023","journal-title":"IEEE Trans. Image Proc"},{"key":"B6","doi-asserted-by":"publisher","first-page":"591","DOI":"10.1109\/TIP.2022.3231135","article-title":"Redfeat: Recoupling detection and description for multimodal feature learning","volume":"32","author":"Deng","year":"2022","journal-title":"IEEE Trans. Image Proc"},{"key":"B7","doi-asserted-by":"publisher","first-page":"94","DOI":"10.1016\/j.isprsjprs.2020.01.013","article-title":"ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data","volume":"162","author":"Diakogiannis","year":"2020","journal-title":"ISPRS J. Photogrammet. Remote Sens"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46493-0_38","article-title":"\u201cIdentity mappings in deep residual networks,\u201d","author":"He","year":"2016","journal-title":"Proceedings of the European Conference on Computer Vision"},{"key":"B9","doi-asserted-by":"publisher","first-page":"113","DOI":"10.1080\/19479832.2019.1707720","article-title":"An overview of deep learning methods for image registration with focus on feature-based approaches","volume":"11","author":"Kuppala","year":"2020","journal-title":"Int. J. Image Data Fusion"},{"key":"B10","doi-asserted-by":"publisher","first-page":"3243","DOI":"10.3390\/rs15133243","article-title":"A coarse-to-fine feature match network using transformers for remote sensing image registration","volume":"15","author":"Liang","year":"2023","journal-title":"Remote Sensing"},{"key":"B11","doi-asserted-by":"publisher","first-page":"3379","DOI":"10.3390\/rs15133379","article-title":"Homography matrix-based local motion consistent matching for remote sensing images","volume":"15","author":"Liu","year":"2023","journal-title":"Remote Sensing"},{"key":"B12","doi-asserted-by":"publisher","first-page":"5892","DOI":"10.1109\/TIP.2024.3472491","article-title":"Grid: Guided refinement for detector-free multimodal image matching","volume":"33","author":"Liu","year":"2024","journal-title":"IEEE Trans. Image Proc"},{"key":"B13","doi-asserted-by":"publisher","first-page":"4721","DOI":"10.1109\/TIP.2025.3586507","article-title":"AU-Net: Adaptive unified network for joint multi-modal image registration and fusion","volume":"34","author":"Lu","year":"2025","journal-title":"IEEE Trans. Image Proc"},{"key":"B14","first-page":"1","article-title":"\u201cRegistration and fusion of visible light and irt images in neurosurgery,\u201d","author":"Moshaei-Nezhad","year":"2021","journal-title":"Proceedings of the IEEE Biomedical Circuits and Systems Conference (BioCAS)"},{"key":"B15","doi-asserted-by":"publisher","first-page":"105209","DOI":"10.1016\/j.imavis.2024.105209","article-title":"UTR: A UNet-like transformer for efficient unsupervised medical image registration","volume":"150","author":"Qiu","year":"2024","journal-title":"Image Vis. Comput"},{"key":"B16","doi-asserted-by":"publisher","first-page":"332","DOI":"10.1016\/j.patrec.2021.08.032","article-title":"Triple-input-unsupervised neural networks for deformable image registration","volume":"151","author":"Qu","year":"2021","journal-title":"Pattern Recognit. Lett"},{"key":"B17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TGRS.2022.3187015","article-title":"Deep feature correlation learning for multi-modal remote sensing image registration","volume":"60","author":"Quan","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens"},{"key":"B18","doi-asserted-by":"publisher","first-page":"101540","DOI":"10.1016\/j.imu.2024.101540","article-title":"Medical image registration in the era of transformers: a recent review","volume":"49","author":"Ramadan","year":"2024","journal-title":"Inform. Med. Unlocked"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2412.19412","author":"Ren","year":"2025","journal-title":"Minima: Modality Invariant Image Matching"},{"key":"B20","doi-asserted-by":"publisher","first-page":"102612","DOI":"10.1016\/j.media.2022.102612","article-title":"Cross-modal attention for multi-modal image registration","volume":"82","author":"Song","year":"2022","journal-title":"Med. Image Anal"},{"key":"B21","doi-asserted-by":"publisher","first-page":"61469","DOI":"10.1109\/ACCESS.2025.3557628","article-title":"MelanoDet: Multimodal imaging system for screening and analysis of cutaneous melanoma","volume":"13","author":"Sultana","year":"2025","journal-title":"IEEE Access"},{"key":"B22","first-page":"77","article-title":"Structure from motion using unordered sets of images","volume":"2015","author":"Tapu","year":"2015","journal-title":"UPB Scientific Bulletin Series C"},{"key":"B23","first-page":"4275","article-title":"\u201cXoFTR: Cross-modal feature matching transformer,\u201d","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Tuzcuoglu","year":"2024"},{"key":"B24","year":"2015"},{"key":"B25","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/ADICS58448.2024.10533619","article-title":"\u201cYOLOv8: a novel object detection algorithm with enhanced performance and robustness,\u201d","author":"Varghese","year":"2024","journal-title":"Proceedings of the International Conference on Advances in Data Engineering and Intelligent Computing Systems (ADICS"},{"key":"B26","doi-asserted-by":"publisher","first-page":"63919","DOI":"10.1007\/s11042-023-17991-2","article-title":"Multimodal image registration techniques: a comprehensive survey","volume":"83","author":"Velesaca","year":"2024","journal-title":"Multimed. Tools Appl"},{"key":"B27","first-page":"740","article-title":"\u201cModeT: Learning deformable image registration via motion decomposition transformer,\u201d","author":"Wang","year":"2023","journal-title":"Proceedings of MICCAI"},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2503.16185","author":"Wu","year":"2025","journal-title":"MapGlue: Multimodal Remote Sensing Image Matching"},{"key":"B29","first-page":"19679","article-title":"\u201cRFNet: Unsupervised network for mutually reinforcing multi-modal image registration and fusion,\u201d","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Xu","year":"2022"},{"key":"B30","doi-asserted-by":"publisher","first-page":"12148","DOI":"10.1109\/TPAMI.2023.3283682","article-title":"Murf: Mutually reinforcing multi-modal image registration and fusion","volume":"45","author":"Xu","year":"2023","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B31","doi-asserted-by":"publisher","first-page":"38544","DOI":"10.1109\/ACCESS.2018.2853100","article-title":"Multi-temporal remote sensing image registration using deep convolutional features","volume":"6","author":"Yang","year":"2018","journal-title":"IEEE Access"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2026.1807482\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,25]],"date-time":"2026-05-25T06:15:48Z","timestamp":1779689748000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2026.1807482\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,25]]},"references-count":31,"alternative-id":["10.3389\/frai.2026.1807482"],"URL":"https:\/\/doi.org\/10.3389\/frai.2026.1807482","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,25]]},"article-number":"1807482"}}