{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T11:39:36Z","timestamp":1782301176189,"version":"3.54.5"},"reference-count":41,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2024,6,11]],"date-time":"2024-06-11T00:00:00Z","timestamp":1718064000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"The Natural Sciences and Engineering Council of Canada (NSERC), CREATE-oN DuTy Program","award":["496439-2017"],"award-info":[{"award-number":["496439-2017"]}]},{"name":"Canada Research Chair in Multipolar Infrared Vision (MIVIM)","award":["496439-2017"],"award-info":[{"award-number":["496439-2017"]}]},{"name":"Canada Foundation for Innovation","award":["496439-2017"],"award-info":[{"award-number":["496439-2017"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Efficient image stitching plays a vital role in the Non-Destructive Evaluation (NDE) of infrastructures. An essential challenge in the NDE of infrastructures is precisely visualizing defects within large structures. The existing literature predominantly relies on high-resolution close-distance images to detect surface or subsurface defects. While the automatic detection of all defect types represents a significant advancement, understanding the location and continuity of defects is imperative. It is worth noting that some defects may be too small to capture from a considerable distance. Consequently, multiple image sequences are captured and processed using image stitching techniques. Additionally, visible and infrared data fusion strategies prove essential for acquiring comprehensive information to detect defects across vast structures. Hence, there is a need for an effective image stitching method appropriate for infrared and visible images of structures and industrial assets, facilitating enhanced visualization and automated inspection for structural maintenance. This paper proposes an advanced image stitching method appropriate for dual-sensor inspections. The proposed image stitching technique employs self-supervised feature detection to enhance the quality and quantity of feature detection. Subsequently, a graph neural network is employed for robust feature matching. Ultimately, the proposed method results in image stitching that effectively eliminates perspective distortion in both infrared and visible images, a prerequisite for subsequent multi-modal fusion strategies. Our results substantially enhance the visualization capabilities for infrastructure inspection. Comparative analysis with popular state-of-the-art methods confirms the effectiveness of the proposed approach.<\/jats:p>","DOI":"10.3390\/s24123778","type":"journal-article","created":{"date-parts":[[2024,6,11]],"date-time":"2024-06-11T12:11:00Z","timestamp":1718107860000},"page":"3778","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Advanced Image Stitching Method for Dual-Sensor Inspection"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9448-803X","authenticated-orcid":false,"given":"Sara","family":"Shahsavarani","sequence":"first","affiliation":[{"name":"Computer Vision and Systems Laboratory (CVSL), Department of Electrical and Computer Engineering, Faculty of Science and Engineering, Laval University, Quebec City, QC G1V 0A6, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9458-9714","authenticated-orcid":false,"given":"Fernando","family":"Lopez","sequence":"additional","affiliation":[{"name":"TORNGATS, 200 Boul. du Parc-Technologique, Quebec City, QC G1P 4S3, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0198-7439","authenticated-orcid":false,"given":"Clemente","family":"Ibarra-Castanedo","sequence":"additional","affiliation":[{"name":"Computer Vision and Systems Laboratory (CVSL), Department of Electrical and Computer Engineering, Faculty of Science and Engineering, Laval University, Quebec City, QC G1V 0A6, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8777-2008","authenticated-orcid":false,"given":"Xavier P. V.","family":"Maldague","sequence":"additional","affiliation":[{"name":"Computer Vision and Systems Laboratory (CVSL), Department of Electrical and Computer Engineering, Faculty of Science and Engineering, Laval University, Quebec City, QC G1V 0A6, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,6,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Kot, P., Muradov, M., Gkantou, M., Kamaris, G.S., Hashim, K., and Yeboah, D. (2021). Recent advancements in non-destructive testing techniques for structural health monitoring. Appl. Sci., 11.","DOI":"10.3390\/app11062750"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Fuentes, S., Tongson, E., and Gonzalez Viejo, C. (2021). Urban green infrastructure monitoring using remote sensing from integrated visible and thermal infrared cameras mounted on a moving vehicle. Sensors, 21.","DOI":"10.3390\/s21010295"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Chen, J., Li, Z., Peng, C., Wang, Y., and Gong, W. (2022). UAV Image Stitching Based on Optimal Seam and Half-Projective Warp. Remote Sens., 14.","DOI":"10.3390\/rs14051068"},{"key":"ref_4","first-page":"255","article-title":"Multi-modal image processing pipeline for NDE of structures and industrial assets","volume":"Volume 12536","author":"Shahsavarani","year":"2023","journal-title":"Thermosense: Thermal Infrared Applications XLV"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Ghiass, R.S., Arandjelovi\u0107, O., Bendada, H., and Maldague, X. (2014). A unified framework for thermal face recognition. Neural Information Processing, Proceedings of the 21st International Conference, ICONIP 2014, Kuching, Malaysia, 3\u20136 November 2014, Springer International Publishing. Proceedings, Part II.","DOI":"10.1007\/978-3-319-12640-1_41"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Ghiass, R.S., Arandjelovi\u0107, O., Bendada, H., and Maldague, X. (2013, January 4\u20139). Illumination-invariant face recognition from a single image across extreme pose using a dual dimension AAM ensemble in the thermal infrared spectrum. Proceedings of the 2013 International Joint Conference on Neural Networks (IJCNN), Dallas, TX, USA.","DOI":"10.1109\/IJCNN.2013.6707095"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Nooralishahi, P., Ramos, G., Pozzer, S., Ibarra-Castanedo, C., Lopez, F., and Maldague, X.P. (2022). Texture analysis to enhance drone-based multi-modal inspection of structures. Drones, 6.","DOI":"10.3390\/drones6120407"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Yi, K.M., Trulls, E., Lepetit, V., and Fua, P. (2016, January 8\u201316). Lift: Learned invariant feature transform. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46466-4_28"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Zhang, L., and Rusinkiewicz, S. (2018, January 18\u201323). Learning to detect features in texture images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00662"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"DeTone, D., Malisiewicz, T., and Rabinovich, A. (2018, January 18\u201322). Superpoint: Self-supervised interest point detection and description. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPRW.2018.00060"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Lenc, K., and Vedaldi, A. (2016, January 8\u201310). Learning covariant feature detectors. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-49409-8_11"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Savinov, N., Seki, A., Ladicky, L., Sattler, T., and Pollefeys, M. (2017, January 21\u201326). Quad-networks: Unsupervised learning to rank for interest point detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.418"},{"key":"ref_13","unstructured":"Ono, Y., Trulls, E., Fua, P., and Yi, K.M. (2018, January 3\u20138). LF-NET: Learning local features from images. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Georgakis, G., Karanam, S., Wu, Z., Ernst, J., and Koseck\u00e1, J. (2018, January 18\u201323). End-to-end learning of keypoint detector and descriptor for pose invariant 3D matching. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00210"},{"key":"ref_15","unstructured":"Barroso-Laguna, A., Riba, E., Ponsa, D., and Mikolajczyk, K. (November, January 27). Key.net: Keypoint detection by handcrafted and learned CNN filters. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Verdie, Y., Yi, K., Fua, P., and Lepetit, V. (2015, January 7\u201312). Tilde: A temporally invariant learned detector. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7299165"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"977","DOI":"10.1016\/S0262-8856(03)00137-9","article-title":"Image registration methods: A survey","volume":"21","author":"Zitova","year":"2003","journal-title":"Image Vis. Comput."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1016\/j.media.2017.07.005","article-title":"A survey on deep learning in medical image analysis","volume":"42","author":"Litjens","year":"2017","journal-title":"Med. Image Anal."},{"key":"ref_19","unstructured":"Dawn, S., Saxena, V., and Sharma, B. (July, January 30). Remote sensing image registration techniques: A survey. Proceedings of the International Conference on Image and Signal Processing, Quebec City, QC, Canada."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1153","DOI":"10.1109\/TMI.2013.2265603","article-title":"Deformable medical image registration: A survey","volume":"32","author":"Sotiras","year":"2013","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1016\/j.media.2017.04.010","article-title":"Slice-to-volume medical image registration: A survey","volume":"39","author":"Ferrante","year":"2017","journal-title":"Med. Image Anal."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"586","DOI":"10.1287\/mnsc.9.4.586","article-title":"The quadratic assignment problem","volume":"9","author":"Lawler","year":"1963","journal-title":"Manag. Sci."},{"key":"ref_23","unstructured":"Belongie, S., Malik, J., and Puzicha, J. (2000, January 1). Shape context: A new descriptor for shape matching and object recognition. Proceedings of the Advances in Neural Information Processing Systems, Denver, CO, USA."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"657","DOI":"10.1016\/j.ejor.2005.09.032","article-title":"A survey for the quadratic assignment problem","volume":"176","author":"Loiola","year":"2007","journal-title":"Eur. J. Oper. Res."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1016\/j.jvcir.2016.06.008","article-title":"A robust iterative super-resolution mosaicking algorithm using an adaptive and directional Huber-Markov regularization","volume":"40","author":"Ghosh","year":"2016","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1007\/s11263-006-0002-3","article-title":"Automatic panoramic image stitching using invariant features","volume":"74","author":"Brown","year":"2007","journal-title":"Int. J. Comput. Vis."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","article-title":"Distinctive Image Features from Scale-Invariant Keypoints","volume":"60","author":"Lowe","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1145\/358669.358692","article-title":"Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography","volume":"24","author":"Fischler","year":"1981","journal-title":"Commun. ACM"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Lin, W.Y., Liu, S., Matsushita, Y., Ng, T.T., and Cheong, L.F. (2011, January 20\u201325). Smoothly varying affine stitching. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995314"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Hafeez, J., Lee, J., Kwon, S., Ha, S., Hur, G., and Lee, S. (2020). Evaluating feature extraction methods with synthetic noise patterns for image-based modelling of texture-less objects. Remote Sens., 12.","DOI":"10.3390\/rs12233886"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Sun, J., Shen, Z., Wang, Y., Bao, H., and Zhou, X. (2021, January 20\u201325). LoFTR: Detector-free local feature matching with transformers. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00881"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1016\/j.neucom.2015.08.104","article-title":"Auto-encoder based dimensionality reduction","volume":"184","author":"Wang","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_33","unstructured":"Tschannen, M., Bachem, O., and Lucic, M. (2018). Recent advances in autoencoder-based representation learning. arXiv."},{"key":"ref_34","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_35","unstructured":"Liu, W., Wen, Y., Yu, Z., and Yang, M. (2016, January 19\u201324). Large-Margin Softmax Loss for Convolutional Neural Networks. Proceedings of the International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Neubeck, A., and Van Gool, L. (2006, January 20\u201324). Efficient Non-Maximum Suppression. Proceedings of the 18th International Conference on Pattern Recognition (ICPR\u201906), Hong Kong, China.","DOI":"10.1109\/ICPR.2006.479"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Cao, S.Y., Hu, J., Sheng, Z., and Shen, H.L. (2022, January 19\u201324). Iterative Deep Homography Estimation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00192"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., and Zitnick, C.L. (2014, January 6\u201312). Microsoft COCO: Common Objects in Context. Proceedings of the Computer Vision\u2014ECCV 2014: 13th European Conference, Zurich, Switzerland. Proceedings.","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Sarlin, P.E., DeTone, D., Malisiewicz, T., and Rabinovich, A. (2020, January 13\u201319). Superglue: Learning feature matching with graph neural networks. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00499"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Rublee, E., Rabaud, V., Konolige, K., and Bradski, G. (2011, January 6\u201313). ORB: An efficient alternative to SIFT or SURF. Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain.","DOI":"10.1109\/ICCV.2011.6126544"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Alcantarilla, P.F., Bartoli, A., and Davison, A.J. (2012, January 7\u201313). KAZE features. Proceedings of the Computer Vision\u2014ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy.","DOI":"10.1007\/978-3-642-33783-3_16"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/12\/3778\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:56:41Z","timestamp":1760108201000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/12\/3778"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,11]]},"references-count":41,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2024,6]]}},"alternative-id":["s24123778"],"URL":"https:\/\/doi.org\/10.3390\/s24123778","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,11]]}}}