{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:49:44Z","timestamp":1760240984971,"version":"build-2065373602"},"reference-count":32,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2019,10,23]],"date-time":"2019-10-23T00:00:00Z","timestamp":1571788800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Centre of Research and Development","award":["POIR 04.01.04-00-0105\/16-00"],"award-info":[{"award-number":["POIR 04.01.04-00-0105\/16-00"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Unmanned aerial vehicles (UAVs) are widely used to protect critical infrastructure objects, and they are most often equipped with one or more RGB cameras and, sometimes, with a thermal imaging camera as well. To obtain as much information as possible from them, they should be combined or fused. This article presents a situation in which data from RGB (visible, VIS) and thermovision (infrared, IR) cameras and 3D data have been combined in a common coordinate system. A specially designed calibration target was developed to enable the geometric calibration of IR and VIS cameras in the same coordinate system. 3D data are compatible with the VIS coordinate system when the structure from motion (SfM) algorithm is used. The main focus of this article is to provide the spatial coherence between these data in the case of relative camera movement, which usually results in a miscalibration of the system. Therefore, a new algorithm for the detection of sensor system miscalibration, based on phase correlation with automatic calibration correction in real time, is introduced.<\/jats:p>","DOI":"10.3390\/rs11212469","type":"journal-article","created":{"date-parts":[[2019,10,25]],"date-time":"2019-10-25T03:20:36Z","timestamp":1571973636000},"page":"2469","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Online Correction of the Mutual Miscalibration of Multimodal VIS\u2013IR Sensors and 3D Data on a UAV Platform for Surveillance Applications"],"prefix":"10.3390","volume":"11","author":[{"given":"Piotr","family":"Sieka\u0144ski","sequence":"first","affiliation":[{"name":"Institute of Micromechanics and Photonics, Warsaw University of Technology, 8 \u015aw. A. Boboli St., 02-525 Warsaw, Poland"}]},{"given":"S\u0142awomir","family":"Pa\u015bko","sequence":"additional","affiliation":[{"name":"Institute of Micromechanics and Photonics, Warsaw University of Technology, 8 \u015aw. A. Boboli St., 02-525 Warsaw, Poland"}]},{"given":"Krzysztof","family":"Malowany","sequence":"additional","affiliation":[{"name":"Institute of Micromechanics and Photonics, Warsaw University of Technology, 8 \u015aw. A. Boboli St., 02-525 Warsaw, Poland"}]},{"given":"Marcin","family":"Malesa","sequence":"additional","affiliation":[{"name":"Institute of Micromechanics and Photonics, Warsaw University of Technology, 8 \u015aw. A. Boboli St., 02-525 Warsaw, Poland"}]}],"member":"1968","published-online":{"date-parts":[[2019,10,23]]},"reference":[{"key":"ref_1","unstructured":"J\u00f3zwik, M., Jaroszewicz, L.R., and Kujawi\u0144ska, M. (2018). The method of acquiring and processing 3D data from drones. Speckle 2018: VII International Conference on Speckle Metrology, SPIE."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"305","DOI":"10.1017\/S096249291700006X","article-title":"A survey of structure from motion","volume":"26","author":"Voroninski","year":"2017","journal-title":"Acta Numer."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Maes, W.H., Huete, A.R., and Steppe, K. (2017). Optimizing the processing of UAV-based thermal imagery. Remote Sens., 9.","DOI":"10.3390\/rs9050476"},{"key":"ref_4","first-page":"130","article-title":"Comparison of four UAV georeferencing methods for environmental monitoring purposes focusing on the combined use with airborne and satellite remote sensing platforms","volume":"75","author":"Planas","year":"2019","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"4003","DOI":"10.3390\/rs6054003","article-title":"Spatial co-registration of ultra-high resolution visible, multispectral and thermal images acquired with a micro-UAV over antarctic moss beds","volume":"6","author":"Turner","year":"2014","journal-title":"Remote Sens."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1007\/978-3-642-32384-3_18","article-title":"Real-time image fusion monitoring system: Problems and solutions","volume":"Volume 184","author":"Putz","year":"2013","journal-title":"Advances in Intelligent Systems and Computing"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"4514","DOI":"10.1364\/AO.54.004514","article-title":"Image registration under translation and rotation in two-dimensional planes using Fourier slice theorem","volume":"54","author":"Pohit","year":"2015","journal-title":"Appl. Opt."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"773","DOI":"10.1109\/ICOSP.2002.1181170","article-title":"A wavelet-based multisensor image registration algorithm","volume":"Volume 1","author":"Huang","year":"2002","journal-title":"Proceedings of the 6th International Conference on Signal Processing (ICSP)"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Li, H., Ding, W., Cao, X., and Liu, C. (2017). Image registration and fusion of visible and infrared integrated camera for medium-altitude unmanned aerial vehicle remote sensing. Remote Sens., 9.","DOI":"10.3390\/rs9050441"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Huang, Q., Yang, J., Wang, C., Chen, J., and Meng, Y. (2012, January 22\u201327). Improved registration method for infrared and visible remote sensing image using NSCT and SIFT. Proceedings of the International Geoscience and Remote Sensing Symposium (IGARSS), Munich, Germany.","DOI":"10.1109\/IGARSS.2012.6351020"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1179\/1743131X12Y.0000000046","article-title":"Multi-spectral image registration and evaluation based on edge-enhanced MSER","volume":"62","author":"Liu","year":"2013","journal-title":"Imaging Sci. J."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1016\/j.isprsjprs.2014.07.015","article-title":"A fast and mobile system for registration of low-altitude visual and thermal aerial images using multiple small-scale UAVs","volume":"104","author":"Yahyanejad","year":"2015","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"3789570","DOI":"10.1155\/2016\/3789570","article-title":"A registration scheme for multispectral systems using phase correlation and scale invariant feature matching","volume":"2016","author":"Li","year":"2016","journal-title":"J. Sensors"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","article-title":"Distinctive image features from scale-invariant keypoints","volume":"60","author":"Lowe","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1462","DOI":"10.3390\/rs4051462","article-title":"Sensor Correction of a 6-Band Multispectral Imaging Sensor for UAV Remote Sensing","volume":"4","author":"Kelcey","year":"2012","journal-title":"Remote Sens."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1330","DOI":"10.1109\/34.888718","article-title":"A flexible new technique for camera calibration","volume":"22","author":"Zhang","year":"2000","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"015402","DOI":"10.1088\/0957-0233\/23\/1\/015402","article-title":"High performance grid for the metric calibration of thermographic cameras","volume":"23","author":"Armesto","year":"2012","journal-title":"Meas. Sci. Technol."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1007\/978-94-007-6639-6_2","article-title":"Geometric calibration of thermographic cameras","volume":"Volume 17","author":"Luhmann","year":"2013","journal-title":"Remote Sensing and Digital Image Processing"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1016\/j.measurement.2017.08.027","article-title":"Highly accurate geometric calibration for infrared cameras using inexpensive calibration targets","volume":"112","author":"Usamentiaga","year":"2017","journal-title":"Measurement"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"7","DOI":"10.2352\/ISSN.2470-1173.2017.11.IMSE-078","article-title":"Accurate Joint Geometric Camera Calibration of Visible and Far-Infrared Cameras","volume":"2017","author":"Shibata","year":"2017","journal-title":"Electron. Imaging"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1625","DOI":"10.1109\/TIM.2012.2182851","article-title":"A mask-based approach for the geometric calibration of thermal-infrared cameras","volume":"61","author":"Vidas","year":"2012","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_22","first-page":"4718","article-title":"Improving calibration of thermal stereo cameras using heated calibration board","volume":"Volume 2015","author":"Saponaro","year":"2015","journal-title":"Proceedings of the International Conference on Image Processing (ICIP)"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1016\/j.infrared.2011.01.002","article-title":"Calibration and verification of thermographic cameras for geometric measurements","volume":"54","author":"Armesto","year":"2011","journal-title":"Infrared Phys. Technol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"3797","DOI":"10.1109\/JLT.2011.2170812","article-title":"Geometric calibration of IR camera using trinocular vision","volume":"29","author":"Yang","year":"2011","journal-title":"J. Light. Technol."},{"key":"ref_25","first-page":"372","article-title":"Multimodal image registration with applications to image fusion","volume":"Volume 1","author":"Heather","year":"2005","journal-title":"Proceedings of the 2005 7th International Conference on Information Fusion"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1007\/s11263-006-0002-3","article-title":"Automatic panoramic image stitching using invariant features","volume":"74","author":"Brown","year":"2007","journal-title":"Int. J. Comput. Vis."},{"key":"ref_27","first-page":"120","article-title":"The OpenCV Library","volume":"25","author":"Bradski","year":"2000","journal-title":"Dr. Dobb\u2019s J. Softw. Tools"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Yahyanejad, S., Quaritsch, M., and Rinner, B. (2011, January 17\u201318). Incremental, orthorectified and loop-independent mosaicking of aerial images taken by micro UAVs. Proceedings of the ROSE 2011-IEEE International Symposium on Robotic and Sensors Environments, Montreal, QC, Canada.","DOI":"10.1109\/ROSE.2011.6058531"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1109\/TPAMI.1986.4767851","article-title":"A Computational Approach to Edge Detection","volume":"PAMI-8","author":"Canny","year":"1986","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_30","unstructured":"Fang, M., Yue, G., and Yu, Q. (2009, January 21\u201323). The study on an application of otsu method in canny operator. Proceedings of the 2009 International Symposium on Information Processing (ISIP 2009), Huangshan, China."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1858","DOI":"10.1109\/TPAMI.2008.113","article-title":"Parametric Image Alignment Using Enhanced Correlation Coefficient Maximization","volume":"30","author":"Evangelidis","year":"2008","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1145\/358669.358692","article-title":"Random sample consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography","volume":"24","author":"Fischler","year":"1981","journal-title":"Commun. ACM"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/11\/21\/2469\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:28:47Z","timestamp":1760189327000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/11\/21\/2469"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,10,23]]},"references-count":32,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2019,11]]}},"alternative-id":["rs11212469"],"URL":"https:\/\/doi.org\/10.3390\/rs11212469","relation":{},"ISSN":["2072-4292"],"issn-type":[{"type":"electronic","value":"2072-4292"}],"subject":[],"published":{"date-parts":[[2019,10,23]]}}}